AD-A156  169 


UNCLASSIFIED 


INDEPENDENT  RESEARCH  AND  INDEPENDENT  EXPLORATORV 
DEVELOPMENT  AT  THE  NAVV.  .  <U)  NAVV  PERSONNEL  RESEARCH 
AND  DEVELOPMENT  CENTER  SAN  DIEGO  CA  R  PENN  ET  AL. 
APR  85  NPRDC-TR-85-23  F/G  5/9 


m  FILE  COPY  AD-A156  169 


I 


NPRDC  TR  85-23 


May  1985 


INDEPENDENT  RESEARCH  AND  INDEPENDENT  EXPLORATORY  DEVELOPMENT 
AT  THE  NAVY  PERSONNEL  RESEARCH  AND  DEVELOPMENT  CENTER — FYSB 


Robert  Penn 
Alice  Crawford 
(Editors) 


Reviewed  by 
James  W.  Tweeddale 


Released  by 
3.  E.  Kohler 
Commander,  U.S.  Navy 
Commanding  Officer 


Acoessien  Tor  .. 

Ttis  graai  Jaf 

DTIC  TAB 

Unannounced  O 

Justification - 

By - - - — - 

Distribution/ _ _ 

Availability  Codes 
~  Avail  and/or 
Dist  Special 


/ 


Navy  Personnel  Research  and  Development  Center 
San  Diego,  California  92152-6800 


Copy 

!^mwm 


•  -  *  v  *  s  • 

*  .  '  *  *  *  *  • 


/-'N  l  ’v  v-v-v-v y  y  v  v-  y  •' 


UlNN^L-rVOJM  IL-LJ 


SECURITY  CiASStfiCATiON  OP  this  f>A06 


REPORT  DOCUMENTATION  PAGE 


la  REPORT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 


2a  SECURITY  CLASSIFICATION  AUTHORITY 


2b  DECLASSIFICATION /DOWNGRADING  SCHEDULE 


4  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 

NPRDC  TR  85- 


1b.  RESTRICTIVE  MARKINGS 


3  DISTRIBUTION /AVAILABILITY  OF  REPORT 

Limited  to  U.S.  Government  agencies  only, 
administrative  or  operational  purposes. 
Refer  reauests  to  NPRDC  Code  04. 


S.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a  NAME  OF  PERFORMING  ORGANIZATION 

Navy  Personnel  Research  and 
Development  Center 


6c  ADDRESS  {City,  State,  and  ZIP  Code) 

San  Diego,  CA  92152-6800 


8a.  NAME  OF  FUNDING /SPONSORING 
ORGANIZATION 

NAVMAT 


8c  ADDRESS  (C/ty,  State,  and  ZIP  Code) 


6b  OFFICE  SYMBOL  I  7a.  NAME  OF  MONITORING  ORGANIZATION 
(If  applicable)  f 


7b  ADDRESS  (City,  State,  and  ZIP  Code) 


8b  OFFICE  SYMBOL  9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(if  applicable)  j 


’0  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM  I  PROJECT 

ELEMENT  NO  |  NO 


62766N 


61152N 


WORK  UNIT 
ACCESSION  NO 


11  title  (include  Security  Classification) 

INDEPENDENT  RESEARCH  AND  INDEPENDENT  EXPLORATORY  DEVELOPMENT  AT  THE  NAVY 
PERSONNEL  RESEARCH  AND  DEVELOPMENT  CENTER  --FY84 


12  PERSONAL  AUTHOR(S) 

Penn.  Robert  and  Crawford.  Alice  (Editors 


s  PAGE  COUNT 

65 


17 _ _ COSATI  CODES _ I  18  SUBJECT  TERMS  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

FIELD  |  GROUP  |  SUB-GROUP  | 

Independent  Research  (IR),  Independent  Exploratory 
Development  (IED) 


19  ABSTRACT  ( Continue  on  reverse  if  necessary  and  identify  by  block  number) 

'This  report  provides  synopses  of  FY84  Independent  Research  (IR)  and  Independent  Exploratory 
Development  (IED)  projects,  the  IR/IED  funding  profile,  and  a  list  of  publications  and  presentations  on 
IR/IED  projects.  In  the  IR  section,  new  research  methods  for  measuring  task  difficulty  and  worker 
ability,  for  collecting  problem-solving  strategies,  and  for  collecting  simultaneous  data  on  cognition  and 
emotion  during  task  performance  are  described.  Personnel  testing  is  addressed  through  an  evaluation  of 
new  formulas  for  range  restriction  and  a  polychotymous  model  for  scoring  multiple  choice  items  that 
uses  information  from  wrong  answers.  Potential  training  innovations  are  described  for  concepts  in  basic 
electronics,  memory  for  procedural  tasks,  and  career-role  learning  by  officers.  Computerized  decision 
aids  for  surveillance  tasks  and  opportunities  for  exploiting  the  huge  individual  differences  in  worker 
productivity  were  also  studied.  In  the  IED  section,  manpower  related  projects  include  a  model  for  the 
man-job  match  by  multiple  policy  criteria,  the  integration  of  current  definitions  of  "manpower  supply," 
and  a  market  segmentation  analysis  of  Navy  retention  incentives.  Computerized  aids  for  plain  English  in 
military  documents  and  for  tactical  action  officer  training  were  also  developed  in  FY84  IED. 


20  Distribution  /availability  of  abstract 
fiP'JNCLASSlFIfO/UNL'MlTEO  □  SAME  AS  RPT 


22*  NAME  OF  RESPONSIBLE  INDIVIDUAL 
Penn.  R. 


21  ABSTRACT  SECURITY  CLASSIFICATION 

□  one  users  UNCLASSIFIED 


22b  TELEPHONE  (Include  Area  Code)  22c.  OFFICE  SYMBOL 

(619)  225-6371  04 


DO  FORM  1473, 84  mar 


83  APR  edition  may  be  used  until  exhausted 
All  other  editions  are  obsolete 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


UNCLASSIFIED 


FOREWORD 


Independent  Research  (IR)  and  Independent  Exploratory  Development  (IED)  funds  are 
provided  to  the  Technical  Directors  of  Navy  Laboratories  to  give  them  wide  latitude  to 
perform  innovative,  promising  work  without  the  procedure  of  formal  and  prior  approval 
which  might  delay  normal  funding  authorization. 

The  IR  program  at  the  Navy  Personnel  Research  and  Development  Center  has  been 
active  since  the  Center  was  formed  in  1973.  It  is  funded  under  PE61152N.  The  IED 
program  was  initiated  in  FY76  and  is  funded  under  PE62766N.  The  Center  received  the 
Director  of  Navy  Laboratories  award  for  science  excellence  based  on  the  IR/IED 
corporate  review  conducted  in  1982. 

This  report  is  submitted  to  fulfill  the  requirement  for  an  annual  IR/IED  report 
(NAVMATINST  3920.3 C).  It  provided  summaries  of  selected  FY84  projects,  the  program 
funding  profile,  and  a  list  of  publications  and  presentations  resulting  from  IR/IED  efforts. 


3.  E.  KOHLER  3.  W.  TWEEDDALE 

Commander,  U.S.  Navy  Technical  Director 

Commanding  Officer 


v 


CONTENTS 


Page 

INDEPENDENT  RESEARCH  (IR)  PROGRAM  .  1 

A  Model  for  Measuring  Task  Difficulty  and  Worker  Ability  .  2 

Individual  Differences:  An  Underdeveloped  Opportunity  for 

Military  Psychology .  6 

The  Treatment  of  Restriction  of  Range  in  Navy  Personnel  Research  .  12 

Models  for  Calibrating  Multiple  Choice  Items  .  15 

Enhancing  Understanding  of  Electric  Circuits  .  19 

A  New  Method  For  Collecting  Problem  Solving  Strategies .  23 

Cognitive  and  Emotional  Processing .  25 

Procedural  Tasks:  Will  Teaching  Theory  Enhance  Retention?  .  28 

Acquiring  Skills  from  Work  Experience  .  33 

Models  of  Human  Performance  with  Application  to  Decision  Making  .  35 

INDEPENDENT  EXPLORATORY  DEVELOPMENT  (IED)  PROGRAM  .  40 

Multiple  Criterion  Optimization  Techniques .  41 

The  Self-Defense  Tactical  Trainer  .  44 

Application  of  Market  Segmentation  to  Navy  Retention  Issues  .  47 

Alternative  Military  Manpower  Supply  Methodologies .  51 

Development  of  Computer-Based  Writing  Aids .  52 

APPENDIX  A — IR/IED  PUBLICATIONS  AND  PRESENTATIONS  . A-0 

APPENDIX  B— PROJECTS  THAT  HAVE  TRANSITIONED  .  B-0 


DISTRIBUTION  LIST 


£i 


.-'i 


INDEPENDENT  RESEARCH  (IR)  PROGRAM 


The  Navy  Personnel  Research  and  Development  Center  (NAVPERSRANDCEN)  has 
been  conducting  Independent  Research  (IR)  since  its  establishment  in  1973.  The  program 
provides  an  opportunity  for  Center  personnel  to  conduct  basic  research  in  innovative 
domains  that  may  have  potential  long  term  benefits  for  the  Navy.  Resources  provided  for 
the  program  have  been  used  to  develop  a  variety  of  research  methods,  models,  and 
techniques  within  the  broad  areas  of  training,  manpower  utilization,  organizational 
productivity,  and  human  factors  in  the  design  of  Navy  weapon  systems  and  platforms. 

IR  work  units  for  FY84  and  FY85  are  listed  in  Table  1.  The  publications  and 
presentations  produced  by  this  work  are  listed  in  Appendix  A.  Work  conducted  within 
selected  units  during  FY84  is  described  in  the  remainder  of  this  section. 

Table  1 

Independent  Research  Work  Units  lor  FY84  and  FYS5 


6I132N 

Project  Title 

Principal 

Investigator 

Fundinc  in 

mT  i 

ZROOO-0 1-042-04 
.005 

Models  and  Measures  of  Human 
Performance® 

Dr.  R.  Sorenson 

A V.-  933-6617 

Code  71 

10 

0 

.021 

Treatment  of  Restriction  of  Range 
in  Navy  Personnel  Research® 

Dr.  E.  AM 

AV:  933-2*08 

Code  62 

20 

0 

.024 

042.06 

Models  for  Calibrating  Multiple 

Choice  Items 

3.  Sympson 

AV:  933-4133 
Code  <2 

30 

49 

.041 

Enhancing  Understanding  ol  Electric 
Circuits 

Dr.  V.  Montague 
AV:  933-7100 

Code  05 

30 

30 

.017 

Generalizable  Cognitive  Strategies 

Dr.  M.  Baker 

AV:  933-4955 

Code  51 

*0 

0 

.025 

Cognitive  an<J  Emotional  Processing 

Dr.  B.  McDonald 
AV:  933-4*3* 

Code  31 

31 

15 

.027 

Cognitive  Factors  in  Learning  and 

Retention 

Dr.  3.  EUis 

AV:  933-4*3* 

Code  51 

37 

10 

.028 

0*2.08 

Cognitive  Storage  Mechanisms^ 

Dr.  M.  Baker 

AV:  933-4953 

Code  51 

0 

1* 

.023 

Acquiring  Skill  From  Work  Experience* 

Dr.  R.  Morrison 
AV:  933-4911 

Code  (2 

27 

0 

.03C 

042.09 

Positively  Accelerating  Sharing  Rates 

Dr.  D.  Nebeker 
AV:  933-4933 

Code  72 

0 

25 

.020 

Analysis  of  Cognition  in  Natural  Settings 

Dr,  E.  Hutchins 
AV:  933-42*2 

Code  501 

12 

so 

.022 

Models  of  Human  Performance  With  Appli¬ 
cation  to  Decision  Aiding 

Dr.  F.  Greltzer 
AV:  933-2081 

Code  71 

*5 

0 

.026 

Effects  of  Exercise  and  Nutrition  on 
Vigilance1* 

Dr.  3.  McGrath 
AV:  933-7*2* 
Code  71 

8 

0 

Military  Leadership 

Dr.  L.  Atwater 

AV:  933-4935 

Code  72 

0 

3* 

Relationships  Between  Management  Prac¬ 
tice  and  Organizational  Performance 

Dr.  K.  Crawford 
AV:  933-4933 

Code  72 

0 

35 

Expert  Systems  for  Fault  Diagnosis 

Dr.  Don  Malkoff 
AV.  933-4617 

Code  71 

_0 

330 

» 

350 

‘Completed  at  the  end  of  FY84. 

^Terminated  in  FY85  due  to  reprioritization  of  Center  research  efforts. 
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A  MODEL  FOR  MEASURING  TASK  DIFFICULTY  AND  WORKER  ABILITY1 

Richard  C.  Sorenson 


Background 

In  an  evaluation  of  technological,  organizational,  or  incentive  changes,  it  is  necessary 
to  measure  the  performance  of  individuals  on  specific  job  tasks.  This  is  difficult  when 
individuals  perform  several  different  job  tasks  during  any  given  work  period.  Thus,  models 
are  required  that  allow  us  to  solve  for  parameters  associated  with  task  difficulty  as  well 
as  worker  ability  from  the  number  of  times  tasks  are  performed  and  the  time  spent 
performing  them. 


Development  of  an  Initial  Model 


Suppose  we  have  a  work  situation  in  which  we  have  I  individuals  performing  .1 
different  job  tasks  and  that  we  have  measurements  over  K  weeks.  We  will  let 


=  the  number  of  times  the 
week  k. 


■th 


job  task  was  performed  by  individual  i  during 


t 


ik 


A 


ik 


B 


jk 


=  the  time  spent  by  individual  i  performing  job  tasks  during  week  k. 

=  the  inverse  of  ability  or  proficiency  associated  with  individual  i  during  week 
k.  Ajk  is  the  length  of  time  individual  i  took  to  perform  a  standard  set  of 

tasks  relative  to  the  average  individual  during  week  k. 

=  the  difficulty  associated  with  job  task  j  during  week  k.  B..  is  the  average 

♦K  * 

length  of  time  it  takes  to  perform  the  j  job  task. 


We  have  measures  for  X  and  t,  and  desire  to  solve  for  A  and  B  using  the  relationship: 


3 

l 


A: 


j=l 

We  will  let 
.X 


k°A 


ik 


X...  B., 

ijk  jk 


t; 


ik 


the  1x3  matrix  of  the  X-^s  for  the  k^1  period, 
the  Ixl  diagonal  matrix  of  the  A^s  for  the  k^1  period, 
the  3*^  order  vector  of  the  B^.s  for  the  k^1  period, 
the  I  order  vector  of  the  t^s  for  the  k  period. 


(1) 


lThis  article  presents  one  of  several  models  dealing  with  measuring  task  difficulty 
and  person  ability  developed  under  work  unit  ZR000-01 -042.005:  Models  and  Measures  of 
Human  Performance. 


From  the  above  definitions  and  (1),  we  specify  the  model 


k°A  kX 


B 


kT  =  ke 


(2) 


by  which  we  will  determine  the  individual  parameters,  A,  and  the  task  parameters,  B,  so 
that  the  sum  of  squares  of  the  e  values  in  (2)  is  a  minimum  subject  to  certain  constraints 
on  A  and  B.  We  shall  constrain  A  and  B  to  be  equal  for  different  periods: 


RDA  =  DA  for  all  k  =  1  .  .  .  K  (3) 

kB  =  B  for  all  k  =  1  .  .  .  K  (4) 

and  the  sum  of  the  person  parameters  is  1,  the  number  of  individuals: 


l'DA  =  A'  (5) 

A'l  =  I  (6) 


We  will  solve  for  the  unknown  A  and  the  unknown  B  by  an  iterative  approach  since 
equation  (2)  involves  their  product.  We  let 

k°A  kX  *  kZ  (7> 

and  from  (21  and  (7) 

kz  kB-kT  *  ke  <8) 

The  least  squares  solution  for  kB  in  (8)  can  be  shown  to  be 

kB  =  Wr>  (kz'kT>  <’> 

Because  of  the  constraint  of  (4),  we  set  the  values  of  B  equal  to  the  mean  of  the  values  kB 
as  indicated  in  the  scalar  equation  (10): 


K 

E 

k=l 


(10) 


and  refer  to  the  vector  of  B.  values  as  B.  If  we  denote  .Tas  an  estimate  of .  T, 
(8)  and  (10)  we  have  '  K 


then  from 


(11) 


From  (2),  (7),  and  (ID 

DAk^kT  =  ke  (12) 

where  DAis  a  diagonal  matrix  of  multipliers  to  update  DAfrom  one  iteration  to  another  to 

be  solved  for  by  minimizing  e'e.  Recalling  the  constraints  of  (3)  we  write  (13)  in  scalar 
notation. 


w  A 


Tik  Ai  "  Tik  =  eik 


(13) 


3 


K 

The  solution  for  (13)  to  minimize  I 

k=l 


ik 


is 


A. 

l 


K 

r 

k=l 


K 

1 

k=l 


(14) 


Thus,  we  may  now  update  and  impose  the  constraint  of  (6)  with  the  replacement 
equations 


A,  =  \  A, 


(15) 


A. 

i 


(16) 


Computer  Algorithm 

We  are  now  ready  to  specify  our  iterative  solution: 

1.  Choose  an  initial  estimate  for  D^.  (The  identity  matrix  would  be  appropriate.) 

2.  Calculate  B  by  equations  (7),  (9),  and  (10). 

3.  Calculate  A  by  equations  (11),  (14),  (15),  and  (16). 

4.  Test  to  see  if  any  B  or  any  A  has  been  modified  more  than  a  tolerance  limit,  e, 
and  if  so  return  to  step  2  and  if  not,  calculation  is  completed. 

Application 

The  initial  model  was  evaluated  by  applying  it  to  data  gathered  in  a  government 
agency  that  dealt  with  expenditures  related  to  government  contracts.  In  the  particular 
unit  studied,  the  staff  members  processed  four  different  kinds  of  payments.  A  total  of  21 
employees  were  involved  over  a  period  of  18  weeks.  Thus,  1=21,  3=4,  and  K=18.  The 
iterative  solutions  for  the  vector  of  task  difficulty  parameters,  B,  is  given  in  Table  1. 
Table  2  provides  the  solutions  for  A,  the  ability  parameter.  Iterations  were  continued 
until  the  largest  change  in  an  A  or  B  element  was  less  than  .05.  The  estimates  of  the 
ability  parameters  converge  rapidly.  It  appears  that  some  individuals  perform  tasks  in  as 
little  as  56  percent  of  the  average  time,  while  others  take  67  percent  more  time  than 
average. 
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Table  1 


Iterative  Solution  for  Difficulty  Parameters  for  Four  Tasks 


Task 

Iteration 

1 

2 

3 

4 

5 

6 

7 

8 

9 

1 

91.65 

65.09 

56.82 

53.99 

52.91 

52.49 

52.33 

52.26 

52.24 

2 

14.89 

23.69 

26.69 

27.85 

28.31 

28.49 

28.56 

28.58 

28.59 

3 

20.61 

26.21 

27.23 

27.63 

27.79 

27.85 

27.88 

27.89 

27.89 

4 

24.26 

27.20 

26.48 

26.14 

26.00 

25.95 

25.93 

25.92 

25.92 

Table  2 

Iterative  Solution  for  Ability  Parameters 

Individual 

Iteration 

1 

2 

3 

4 

5 

6 

7 

8 

9 

1 

1.45 

1.51 

1.53 

1.54 

1.54 

1.54 

1.54 

1.54 

1.54 

2 

1.06 

1.03 

1.02 

1.02 

1.01 

1.01 

1.01 

1.01 

1.01 

3 

1.17 

1.23 

1.25 

1.25 

1.26 

1.26 

1.26 

1.26 

1.26 

4 

1.60 

1.64 

1.66 

1.67 

1.67 

1.67 

1.67 

1.67 

1.67 

5 

.61 

.60 

.59 

.59 

.59 

.59 

.59 

.59 

.59 

6 

1.08 

1.05 

1.05 

1.04 

1.04 

1.04 

1.04 

1.04 

1.04 

7 

1.09 

1.08 

1.08 

1.08 

1.08 

1.08 

1.08 

1.08 

1.08 

8 

1.07 

1.07 

1.07 

1.07 

1.07 

1.07 

1.07 

9 

1.09 

1.10 

1.10 

1.10 

1.10 

1. 10 

1.10 

1.10 

10 

.88 

.86 

.85 

.85 

.85 

.85 

.85 

.85 

.85 

11 

1.03 

1.02 

1.02 

1.01 

1.01 

1.01 

1.01 

1.01 

1.01 

12 

.97 

1.00 

1.01 

1.01 

1.01 

1.01 

1.01 

1.01 

1.01 

13 

1.09 

1.07 

1.06 

1.06 

1.06 

1.06 

1.06 

1.06 

1.06 

14 

.90 

.93 

.94 

.94 

.94 

.94 

.94 

.94 

.94 

15 

.79 

.81 

.82 

.82 

.82 

.82 

.82 

.82 

.82 

16 

1.00 

.93 

.91 

.90 

.89 

.89 

.89 

.89 

.89 

17 

.61 

.62 

.63 

.63 

.63 

.63 

.63 

.63 

.63 

18 

.60 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

19 

1 .22 

1.19 

1.19 

1.18 

1.18 

1. 18 

1.18 

1.18 

I. IS 

20 

.58 

.56 

.56 

.56 

.56 

.56 

.56 

.56 

.56 

21 

1.14 

1.10 

1.08 

1.08 

1.08 

1.07 

1.07 

1.07 

1.07 

.'-1 
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INDIVIDUAL  DIFFERENCES:  AN  UNDERDEVELOPED  OPPORTUNITY 
FOR  MILITARY  PSYCHOLOGY1 

Bernard  Rimland 

Gerald  E.  Larson 


Background 

Students  of  warfare  have  stated  with  striking  regularity  that  military  victories  have 
depended  more  often  upon  the  quality  of  the  soldiers  than  upon  their  numbers  or  the 
superiority  of  their  weapons.  As  stated  by  Brackney  (1959),  man's  success  has  "depended 
more  on  how  he  has  used  his  weapons  than  upon  any  superiority  of  weapons  design  and 
performance." 

History  shows  many  examples  where  the  tide  of  battle,  and  ultimately  the  tide  of 
war,  has  hinged  on  the  spectacularly  effective  performance  of  just  a  few  individuals.  The 
story  of  Audie  Murphy  is  an  interesting  case  in  point.  Murphy,  who  was  only  5'5"  tall  and 
weighed  just  110  pounds,  was  rejected  by  both  the  Navy  and  Air  Force  before  being 
accepted  for  duty  by  the  Army.  Recounting  even  a  portion  of  his  extraordinary 
performances  would  take  more  space  than  this  article.  He  became  the  most  decorated 
soldier  of  World  War  II,  winning  33  military  awards,  citations,  and  decorations,  including 
every  possible  American  medal  for  valor  as  well  as  three  French  medals  and  one  Belgian 
medal  (Simpson,  1975).  Similarly,  in  World  War  I,  Baron  Von  Richthofen  alone  was 
responsible  for  downing  80  enemy  aircraft  (Carisella  &  Ryan,  1969),  while  an  American, 
Samuel  Woodfill,  single-handedly  overcame  three  German  machine  gun  nests  in  one 
afternoon,  dispatching  21  soldiers  (Blumenson  &  Stokebury,  1975). 

These  are  not  isolated  instances.  In  summarizing  the  findings  of  Brigadier  General 
S.  L.  A.  Marshall,  Toomepuu  (1980)  noted: 

Marshall  has  published  important  findings  on  combat  performance  of 
soldiers  in  World  War  II,  in  Korea,  Viet  Nam,  and  the  Arab/Isreali 
wars.  While  gathering  historical  data  from  front  line  infantry  units 
during  World  War  II,  Marshall  made  the  startling  discovery  that  only 
about  15  percent  of  the  soldiers  in  battle  actually  fired  their 
weapons,  and  that  the  fighters  were  observedly  different  from  the 
other  soldiers.  In  a  bitterly  fought  battle  for  Omaha  Beach,  he  found 
that  on  a  two  division  front  only  six  rifle  companies  could  be 
considered  effective  as  units,  and  only  47  men,  at  widely  scattered 
intervals  along  the  beach,  saved  the  day  from  disaster.  Marshall 
concluded  that  the  outcomes  of  battles  are  decided  by  relatively  few 
effective  participants,  a  conclusion  supported  by  other  astute 
observers  of  the  performance  of  soldiers  in  battle,  (p.  5) 

In  contrast,  a  single  ineffective  performer  in  a  key  position  can  cripple  a  mission. 
Incompetence  on  the  part  of  the  B-29  aircraft  commander,  radar  observer,  or  bombardier 
often  led  to  failure  of  the  entire  mission  (Bass,  1982).  The  outcome  of  the  recent 


lThis  article  summarizes  a  more  lengthy  paper  submitted  for  journal  publication 
prepared  under  work  unit  ZROOO-0 1-042.005:  Models  and  Measures  of  Human  Perfor¬ 
mance. 
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Falkland  Islands  conflict  might  have  been  different  if  the  final  Argentine  Exocet  missile 
had  not  been  mistakenly  wasted  on  a  cargo  ship.  Dixon  (1976),  in  On  the  Psychology  of 
Military  Incompetence,  attributes  many  fiascos  to  defects  in  various  leaders.  But  whether 
one  wishes  to  focus  on  human  ability,  personality,  leadership,  courage,  or  physical 
strength  and  stamina,  it  is  clear  that  individual  differences  play  an  enormously  important 
role  in  the  outcome  of  endeavors.  The  purpose  of  this  paper  is  to  remind  our  readers  of 
the  extent  and  pervasiveness  of  individual  differences,  and  to  offer  suggestions  for 
research  and  applications. 

The  Magnitude  of  Performance  Differences 

The  striking  range  of  differences  in  human  performance  is  apparent  in  every 
dimension  imaginable  (e.g.,  intelligence,  musical  talent,  motivation,  experience,  athletic 
ability).  Rather  than  the  10,  20,  or  even  30  percent  difference  that  most  people  believe 
characterizes  the  span  of  performance,  the  actual  range,  even  in  small  groups,  rarely  falls 
below  200  percent,  and  differences  in  the  thousands  of  percents  are  not  uncommon.  Let 
us  consider  human  performance  differences  in  a  few  fields. 

Academic/Intellectual  Abilities 


Striking  differences  between  individuals  have  been  found  in  studies  of  academic 
performance.  In  terms  of  test  items  correct,  differences  of  several  hundred  percent  (e.g., 
scores  ranges  of  30  to  90+)  are  commonly  found  in  classrooms  of  rather  homogeneous 
college  students.  In  terms  of  scholastic  achievement,  some  high  school  seniors,  about  10 
percent,  surpass  50  percent  of  college  seniors;  in  terms  of  reading  ability,  10  percent  of 
students  in  grade  five  read  better  than  50  percent  of  students  in  grade  eight  (both  cited  in 
Willerman,  1979). 

IQ  scores  usually  range  between  80  and  150  in  a  large  high  school,  but  they  grossly 
understate  the  actual  range  of  difference  in  intelligence.  In  terms  of  number  of  test 
items  correct  (e.g.,  vocabulary,  reasoning,  general  information,  etc.),  the  individual  with 
an  IQ  of  150  outscores  the  individual  with  an  IQ  of  80  by  a  ratio  of  50:  J.  The  mere  2:1 
ratio  in  ability  implied  by  the  IQ  scores  is  misleading.  Consider,  for  example,  the  mental 
ability  of  mathematician  George  Willis  (Wechsler,  (1952).  On  one  occasion  Willis  amused 
himself  by  mentally  extracting  the  square  root  of  a  number  containing  53  digits.  A  month 
latter  he  was  able  to  reproduce  both  the  number  and  the  root. 

Industrial  Productivity 


Individual  differences  in  productivity  are  usually  vastly  underestimated.  When  an 
industrial  supervisor  is  asked  to  estimate  the  difference  between  the  most  and  least 
productive  employee,  he  or  she  will  typically  guess  20  to  30  percent.  However,  when 
actual  production  figures  are  available,  it  turns  out  that  the  average  range  of  differences, 
even  for  simple  tasks,  has  been  about  200  percent. 

If  the  job  is  simple,  such  as  keypunching,  typing,  or  operating  a  sewing  machine, 
productivity  differences  in  the  range  of  200  to  300  percent  are  common  (Wechsler,  1952). 
In  a  recent  review  of  the  literature  on  individual  differences  in  productivity,  Schmidt  and 
Hunter  (1983)  reported  production  data  for  40  groups  of  workers  studied  from  1928  to 
1978.  The  data  were  primarily  from  manual  workers--machine  operators,  card  punchers, 
typists,  assemblers,  etc.  The  productivity  ratios,  derived  by  comparing  the  most  and  least 
productive  five  percent  of  workers,  ranged  between  1.41  and  17.35!  (The  next  lowest 
ratio  below  17.35  was  3.46,  still  an  immense  difference  between  5th  and  95th  percentile 
performers.)  The  average  productivity  ratio  for  simple  tasks  was  about  2.00. 


7 


They  requested  that  these  tests  be  used  to  assist  in  evaluation  of  the  new  materials.  Four 
groups  of  students  (N  =  22-30  each)  who  used  either  the  revised  materials  or  the  standard 
materials  were  given  either  the  CAUSE  or  the  CHANGE  tests  during  Juiy-September 
1984.  One  major  change  to  the  materials  was  the  use  of  test  items  similar  to  those  in  the 
CHANGE  test  in  the  course  pretests. 


Results 

The  data  are  extensive  and  can  receive  only  cursory  review  here.  Of  the  first  group 
of  46  students  who  had  completed  the  14  BE/E  core  modules  successfully,  only  4  made  no 
errors  on  the  CHANGE  tests.  It  is  important  to  recognize  that  these  students  had 
successfully  passed  the  course  and,  in  a  sense,  had  been  certified  as  sufficiently  trained  to 
go  on  to  the  next  course.  Subsequent  administration  of  the  tests  revealed  about  the  same 
high  proportion  of  errors.  Most  students  made  multiple  errors.  Analysis  revealed  that 
there  was  consistency  in  the  errors  made--most  errors  were  concerned  with  misunder¬ 
standings  about  voltage  drop  and  current.  This  finding  is  in  agreement  with  Riley's  (1984) 
results.  Preliminary  analysis  of  the  errors  diagnosed  by  the  CHANGE  test  indicates  that 
consistent  errors  could  be  summarized  by  a  few  "buggy"  (faulty)  rules.  The  intention  for 
use  of  the  results  is  that  an  analysis  of  the  student  buggy  rules  which  cause  the  errors  will 
serve  as  the  basis  for  changes  in  instructional  materials  to  be  tested  during  FY85. 

The  comparison  of  the  groups  receiving  the  revised  or  standard  modules  revealed  no 
overall  differences  in  the  number  or  types  of  errors  in  the  CHANGE  or  CAUSE  tests. 
Again,  errors  revealed  confusion  between  voltage  drop  and  current.  A  more  detailed 
analysis  of  these  data  is  underway  to  examine  any  effects  on  test  items  most  relevant  to 
changes  in  the  instructional  content. 

Conclusions 

The  ongoing  research  and  analysis  allows  no  definitive  conclusions  at  present. 
However,  certain  general  conclusions  seem  reasonable.  The  similarity  of  student 
problem-solving  difficulties  found  by  many  researchers  suggests  that  the  observations  are 
robust  and  valid.  The  difficulty  students  have  in  solving  circuit  problems  seem  to  be  due 
to  imperfect  conceptual  or  mental  modules  of  circuit  functioning.  These  imperfections 
can  be  summarized  by  a  few  buggy  rules,  which  can  be  diagnosed  by  tests.  Future  effort 
should  concentrate  on  refining  the  testing  procedures  and  on  developing  instructional 
approaches  for  correcting  the  buggy  rules. 

Plans  for  Continuing  Studies  in  FY85 

The  analyses  of  the  data  on  the  issues  will  be  combined  into  a  summary  report.  At 
the  request  of  the  CE/I  (curriculum  evaluation/instruction)  group  at  the  BE/E  school, 
NPRDC  will  continue  to  assist  in  assessing  the  new  course  materials.  There  are  three 
main  thrusts  in  the  work  for  FY85:  students'  technical  concepts  prior  to  training; 
diagnostic  testing  refinement,  which  will  be  coordinated  with  the  ONR  testing  laboratory 
effort;  and  course  materials  prepared  specifically  to  overcome  student  difficulties. 
Course  materials  will  be  tested  for  their  effectiveness  in  reducing  or  eliminating  faulty 
concepts  in  June-September  1985. 
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Work  in  this  IR  effort  during  FY83  examined  materials  from  BE/E  course.  Data  on 
student  errors  on  test  items  were  analyzed  to  determine  how  well  the  test  diagnosed  the 
problems  students  had  with  the  concepts  covered  by  the  test  items.  There  were  high 
frequencies  of  errors  on  certain  modules  and  certain  types  of  questions.  The  course  tests, 
however,  were  relatively  inadequate  for  revealing  the  details  of  student  misunderstanding. 
Therefore,  there  is  a  need  to  develop  more  systematic  means  for  assessing  student 
difficulties. 

Objectives 

This  project  was  undertaken  to  (1)  examine  student  performance  and  learning  of  basic 
concepts  of  electricity,  (2)  identify  deficits  and  flaws  in  student  solutions  to  test  problems 
about  simple  circuits,  (3)  discover  reasons  for  the  difficulties  students  have,  and  (4) 
develop  ways  to  overcome  the  difficulties.  The  specific  objective  for  FY84  was  to  assess 
the  student  understanding  of  electricity  and  simple  circuits  by  devising  diagnostic  tests. 
Subsequent  work  will  develop  approaches  to  instruction  that  may  overcome  the  difficul¬ 
ties  found. 

Approach 

Beginning  during  FY83,  the  tests  used  in  the  Navy's  BE/E  course  were  analyzed  to 
determine  whether  problems  in  understanding  could  be  diagnosed  from  the  Progress 
Checks,  Progress  Tests,  and  Comprehensive  Tests  in  use.  This  analysis  carried  over  into 
the  current  effort,  due  in  part  to  changes  in  the  test  items  made  because  of  our 
recommendations. 

Based  on  this  analysis  and  on  an  analysis  by  Riley  (1984),  new  diagnostic  tests  for 
assessing  student  understanding  of  circuits  (competence)  and  for  examining  the  extent  and 
type  of  student  misunderstanding  were  written.  One  test  is  called  the  CHANGE  test. 
Each  item  requires  a  student  to  report  the  effects  of  a  hypothetical  change  in  a  circuit 
for  each  circuit  component.  The  test  is  qualitative,  that  is,  it  requires  the  student  to 
indicate  whether  values  increase,  decrease,  or  stay  tK~  same  given  the  change.  This  test 
represents  an  improvement  over  the  previous  multipie-choice  tests  in  two  ways.  First,  it 
is  qualitative  only,  thus  eliminating  the  errors  in  calculation  which  are  only  indirectly 
relevant  to  circuit  understanding  and  which  confuse  diagnosis.  Second,  because  a  student 
has  to  indicate  what  happens  to  every  circuit  component  or  measure,  patterns  of  errors 
related  to  underlying  misconceptions  (the  faulty  or  "buggy"  rules)  are  detectible.  The 
analysis  of  data  from  the  new  tests  has  concentrated  on  detecting  error  patterns  similar 
to  those  found  by  Riley  (1984)  in  her  analysis  of  student  protocols  taken  while  students 
studied  the  BE/E  materials. 

The  CHANGE  test  was  first  given  to  46  students  in  the  BE/E  course  who  had 
completed  the  14  modules  that  cover  both  DC  and  AC  electric  circuits.  They  were  tested 
individually  or  in  small  groups,  and  were  asked  to  indicate  any  questions  they  could  not 
understand  and  to  comment  on  the  test  itself  by  writing  on  the  back  of  the  pages. 

A  second  new  diagnostic  test,  the  CAUSE  test,  requires  students  to  indicate  the 
possible  causes  for  two  observed  changes  in  a  circuit.  It  presents  a  prototypic  trouble¬ 
shooting  task  and  requires  students  to  use  their  knowledge  of  circuit  functioning  to 
identify  potential  causes  of  observed  changes  in  circuit  values. 


The  BE/E  school  changed  some  of  the  instructional  modules  and  the  format  of  their 
testing  during  the  period  during  which  the  CHANGE  and  CAUSE  tests  were  developed. 


ENHANCING  UNDERSTANDING  OF  ELECTRIC  CIRCUITS1 

William  E.  Montague 


Background 

The  teaching  of  basic  electricity  and  electric  circuits  is  a  primary  task  in  training  for 
jobs  concerned  with  electricity  and  electronics.  The  concepts  of  energy  potential, 
circuits,  resistance,  and  current,  which  are  the  prime  content  of  such  courses,  are 
fundamental  to  learning  about  electronic  devices  and  to  maintaining  such  systems. 
Because  of  the  value  of  these  courses  and  the  difficulties  students  experience  in  learning 
the  material,  research  interest  has  increased  recently.  Studies  have  begun  to  examine  the 
role  of  students'  cognitive  knowledge  structures  in  learning  and  understanding,  the 
adequacy  of  students'  scientific  concepts,  and  how  well  their  concepts  explain  how  devices 
work  or  malfunction. 

The  Navy  teaches  Basic  Electricity  and  Electronics  (BE/E)  to  about  25,000  trainee 
technicians  each  year.  Currently,  the  course  provides  "core"  knowledge  and  general  test 
equipment  skills  before  trainees  attend  about  20  different  follow-on  courses  in  Navy  "A" 
schools.  Similar  large-scale  core  training  is  provided  by  the  other  services.  The  Navy 
course  is  self-paced,  individualized,  and  structured  in  modules.  The  materials  are 
arranged  in  a  simple-to-complex  sequence  in  workbooks  that  students  study  in  individual 
carrils.  The  workbooks  provide  step-by-step  programmed  instruction  pages  and  a 
narrative  covering  the  same  topic.  Interspersed  in  the  materials  are  problems  to  be 
solved  along  with  the  worked  out  answers.  At  the  end  of  each  module  there  is  a  self-test 
(Progress  Check)  with  questions  similar  to  those  that  will  be  asked  on  the  official  test 
(Progress  or  Module  Test).  The  Module  Test  is  presented  on  microfiche  and  processed  by 
the  Navy's  Computer-Managed  Instruction  (CMI)  system,  which  scores  the  test,  records 
student  scores  and  errors,  and  provides  appropriate  remedial  assignments. 

This  course  has  been  plagued  with  high  attrition.  Dissatisfaction  with  student 
knowledge  and  skills  have  also  been  expressed  by  follow-on  schools.  Attempts  to  alleviate 
these  problems  by  revising  course  materials  have  been  made  many  times  over  the  last 
decade,  but  attrition  and  remediation  rates  have  remained  about  the  same.  This  suggests 
that  the  underlying  reasons  for  the  problems  may  be  more  fundamental  to  student  learning 
and  understanding  than  to  course  materials. 

Recent  research  on  the  learning  of  scientific  concepts  and  on  problem  solving 
suggests  that  the  problems  in  such  courses  are  due  to  fundamental  students  difficulties  in 
learning  the  concepts,  inter-relating  them,  and  then  applying  this  knowledge  to  observed 
phenomena  and  working  devices  (i.e.,  describing,  inferring  effects  of  changes  in  compo¬ 
nents,  etc.).  This  might  be  due  to  the  difficulty  of  the  concepts,  the  adequacy  of  the 
instructional  presentations,  or  inadequate  student  prior  knowledge  (or  combinations  of  the 
three).  More  detailed  analysis  of  student  difficulties  should  provide  a  basis  for  more 
principled  course  revision. 
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‘The  FY83  and  FY84  results  of  the  work  under  this  project  title  (ZROOO-0 1-042- 
04.014)  will  provide  the  basis  for  evaluating  the  effects  of  changing  the  delivery  of  the 
Navy  Basic  Electricity  and  Electronic  Course  from  self-paced  to  group-paced/lecture- 
based  instruction.  The  CHANGE  diagnostic  test  developed  for  this  project  is  being  used 
for  an  ONR  supported  research  effort  to  test  the  usefulness  of  adaptive  testing  models  for 
instruction. 
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weights  lead  to  test  scores  that  are  maximally  reliable.  A  new  family  of  distribution 
functions  that  can  be  used  to  model  latent  ability  distributions  in  IRT  was  also  developed 
(Sympson  &  France,  1984).  This  family  of  distribution  functions  is  potentially  useful  for  a 
variety  of  applications  in  statistics  and  psychometrics. 

Plans 

Plans  for  FY85  include  development  of  a  new  algorithm  for  fitting  IRT  models.  This 
new  fitting  procedure  will  require  less  stringent  statistical  assumptions  than  the 
approaches  currently  used.  If  this  effort  is  successful,  the  range  of  applications  in  which 
IRT  models  can  be  used  will  increase  significantly. 

Benefits 


If  polychotomous  IRT  models  are  adopted  for  the  calibration  of  test  items,  two 
immediate  benefits  will  be  realized.  First,  because  fewer  test  items  will  have  to  be 
rejected  for  lack  of  fit  to  the  assumed  IRT  model,  more  items  will  be  available  for  the 
construction  of  new  aptitude  tests.  Second,  the  use  of  information  from  incorrect 
answers  an  examinee  has  selected  will  improve  our  estimates  of  ability.  It  is  estimated 
that  the  use  of  polychotomous  IRT  models  will  allow  us  to  reduce  the  length  of  aptitude 
tests  by  about  15  percent  while  maintaining  test  reliability  at  current  levels. 
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MODELS  FOR  CALIBRATING 
MULTIPLE-CHOICE  ITEMS 
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Polychotomous  scoring  allows  us  to  distinguish  among  people 
who  answer  incorrectly.  This  increases  test  reliability. 


Figure  1.  Models  for  calibrating  multiple-choice  items. 


MODELS  FOR  CALIBRATING  MULTIPLE-CHOICE  ITEMS 


James  Bradford  Sympson 


Background 

The  effectiveness  of  personnel  selection  procedures  that  are  based  on  objective 
measurement  of  individual  aptitudes  and  personality  has  been  well  documented.  Aptitude 
measurement,  in  particular,  has  been  widely  used  in  industry,  the  civil  services,  and  the 
military.  The  Office  of  Naval  Research  has  sponsored  basic  research  on  the  theoretical 
foundations  of  psychological  measurement  for  a  number  of  years.  Recently,  a  new  type  of 
measurement  theory  is  leading  to  a  variety  of  important  advances  in  personnel  testing. 
This  approach  to  test  construction,  scoring,  and  evaluation  is  known  as  item  response 
theory  (Lord,  1980).  Application  of  item  response  theory  allows  more  precise  ability 
estimation,  especially  through  computerized  test  administration. 

The  basic  assumption  of  item  response  theory  (IRT)  is  that  the  probability  of 
selecting  a  particular  response  to  a  test  question  is  related  to  a  person's  level  of  ability 
(or  other  relevant  personal  characteristic)  by  a  specified  mathematical  function.  What 
differentiates  various  IRT  models  is  the  form  of  the  mathematical  function  that  is 
assumed. 

Problem 


Currently-used  IRT  models  classify  responses  to  multiple-choice  test  questions  as 
either  correct  or  incorrect.  These  dichotomous  models  make  no  distinction  among  the 
different  incorrect  responses  a  person  might  select.  Information  about  the  examinee’s 
level  of  ability  that  could  ’  ?  extracted  by  taking  into  account  which  particular  incorrect 
responses  have  been  selected  is  lost  when  these  models  are  used. 

Approach 

In  this  research  project,  polychotomous  IRT  models  are  being  developed. 
Polychotomous  models  keep  the  various  incorrect  responses  to  a  multiple-choice  question 
distinct.  When  an  examinee's  test  is  scored,  these  models  allow  use  of  the  pattern  of 
incorrect  responses,  as  well  as  the  pattern  of  correct  responses,  in  estimating  the 
examinee's  level  of  ability.  Polychotomous  item  scoring  increases  test  reliability  (see 
Figure  1). 

Progress 

A  research  project  initiated  in  FY81  through  IR  funding  resulted  in  the  development 
of  a  new  polychotomous  IRT  model  (Sympson,  1981).  In  FY82  and  FY83,  work  continued 
without  IR  funding.  In  FY82,  an  improved  polychotomous  model  was  developed  (Sympson, 
1982).  A  comparison  of  the  improved  model  with  the  model  developed  in  FY81,  and  with 
polychotomous  models  developed  by  other  researchers  (Bock,  1972;  Samejima,  1979), 
indicated  that  the  improved  model  was  superior  (Sympson,  1983). 

In  FY84,  IR  funds  were  made  available  for  development  of  additional  procedures  and 
models  that  will  be  needed  to  apply  polychotomous  IRT  models  to  test  data.  During  this 
period,  a  computer  program  was  developed  that  computes  scoring  weights  for  the  various 
response  categories  of  multiple-choice  items  (Sympson,  1984).  These  "optimized"  scoring 
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Results 


None  of  the  proposed  new  correction  procedures  is  adequate  for  Navy  selection  and 
classification  problems.  The  procedure  of  Bryant  and  Gokhale  (1972),  for  example, 
requires  knowledge  of  the  criterion  variability  in  an  unselected  applicant  population.  This 
cannot  be  reasonably  obtained  in  a  Navy  setting.  The  remaining  procedures  were  similarly 
inappropriate  or  contained  hidden  assumptions  not  met  in  a  Navy  setting. 

A  finding  of  some  interest  was  that  even  the  simple  three-variable  correction 
formula  that  is  often  used  in  selection  settings  does  not  use  all  the  information  available 
for  deriving  corrected  validities.  A  modified  alternative  formula  was  developed  that  was 
found  to  be  superior  in  Monte  Carlo  evaluations. 

Plans 


Because  no  promising  new  corrections  of  range  restriction  appear  to  exist  in  the 
scientific  literature,  no  further  work  to  evaluate  statistical  correction  formulas  is  planned 
at  present. 


THE  TREATMENT  OF  RESTRICTION  OF  RANGE  IN 
NAVY  PERSONNEL  RESEARCH 


Edward  F.  Alf,  3r. 
Norman  M.  Abrahams 


Background 

A  major  task  in  personnel  research  is  determining  the  effectiveness  of  selection 
devices  such  as  tests,  interviews,  or  background  information.  While  there  are  many  ideal 
research  designs  for  such  evaluations,  there  are  also  many  real-world  prohibitions  on  their 
use. 


In  the  ideal  design,  the  effectiveness,  or  validity,  of  a  selection  device  is  established 
on  a  random  sample  of  applicants.  This  requires  that  a  random  sample  of  applicants  be 
tested,  and  then  admitted  to  training  or  placed  on  the  job  so  that  their  performance  can 
be  measured.  The  effectiveness  of  the  selection  device,  which  is  often  expressed  as  a 
correlation  (validity  coefficient),  is  indicated  by  the  magnitude  of  relation  between  the 
performance  measures  and  the  scores  on  the  selection  device  within  this  group. 

In  practice,  a  random  sample  of  job  applicants  is  rarely  admitted  to  any  program  in 
the  Navy.  Cost,  administrative  infeasibility,  or  other  factors  require  that  only  a 
subsample  that  has  been  screened  for  suitability  is  admitted.  Since  personnel  researchers 
must  evaluate  selection  devices  based  on  results  with  these  preselected  samples,  standard 
statistical  corrections  for  the  "restriction  in  range"  of  the  sample  are  well-known  and 
widely  used.  Without  these  corrections,  the  validity  of  the  selection  device  for  use  on  the 
whole  applicant  population  is  typically  underestimated.  Even  with  such  correction 
procedures,  validity  estimates  are  inaccurate  when  certain  statistical  assumptions  are  not 
met.  Specifically,  the  precise  basis  for  selection  versus  rejection  must  be  known  and 
quantifiable. 

Objective 

Several  new  formulas  for  correcting  selection  device  validity  coefficients  for 
restriction  in  range  of  the  sample  have  recently  been  proposed  (Bryant  &  Gokhale,  1972; 
Hsu,  1982;  Linn,  Harnisch,  fc  Dunbar,  1981;  Olson  &  Becker,  1983;  Wells  &  Fruchter, 
1970).  They  were  mainly  developed  to  circumvent  the  limitations  and  assumptions  of 
conventional  formulas,  for  example,  limitations  on  their  appropriateness  when  subjective 
criteria  for  which  no  quantitative  measures  are  available  have  been  used  to  select  the 
sample.  This  research  reviewed  the  adequacy  of  these  new  formulas  for  use  in  the  Navy 
selection  and  classification  setting  and  examined  their  stability  by  using  Monte  Carlo 
methods. 

Approach 

The  general  approach  was  to  simulate  a  trivariate  normal  distribution  for  a  personnel 
sample  that  had  been  screened  on  one  of  three  variables.  A  computer  program  was 
written  to  model  sampling  from  the  trivariate  normal  distribution  screened  on  one 
variable.  Actual  Navy  population  and  Class  A  School  sample  intercorrelations  were 
examined  to  find  reasonable  parameters  to  use  in  the  model  evaluation.  Criteria  for 
evaluating  the  corrected  selection  instrument  validities  were  developed  based  on  the 
absolute  size  of  the  error  of  estimates  for  the  Navy  population  validity. 
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Conclusions 


The  organizational  implications  of  a  thorough  study  of  individual  differences  are 
immense.  The  recommendations  resulting  from  such  a  study  would  undoubtedly  have  a 
major  impact  on  military  performance  and  on  productivity.  Whatever  the  reasons  for 
overlooking  individual  differences  in  the  past,  there  seems  little  justification  for  failing  to 
recognize  and  use  them  now,  insofar  as  we  can,  to  improve  the  future.  Implementing  the 
ideas  presented  in  this  paper,  and  generating  better  ones,  will  not  be  easy.  However, 
sooner  or  later  it  must  be  done  if  individuals  and  organizations  are  to  reach  their  full 
potential. 


No  one--historian,  biologist,  or  philosopher --has  hitherto  appreciated 
how  important  individuality  is  in  human  affairs  .  .  .  many  have 
recognized  that  individuality  is  highly  important.  But  ...  it  is  even 
more  important  than  the  most  ardent  advocates  would  have 
indicated,  (p.  221) 

Despite  the  enormous  impact  of  the  range  of  human  ability,  managers  and  their 
advisors— including  operations  researchers,  human  factors  specialists,  and  even  psycholo¬ 
gists— continue  to  deal  with  a  hypothetical  "average"  individual  rather  than  the 
differences  between  individuals.  Enormous  gains  would  be  possible  if  organizations  were 
committed  to  identifying  and  optimizing  these  differences.  Here  are  some  important  first 
steps  that  might  be  taken  by  both  military  and  civilian  organizations: 

1.  Personnel  selection,  assignment,  and  promotion.  The  attention  and  resources 
devoted  to  the  measuring  and  using  individual  differences  in  finding  the  right  persons  to 
employ,  placing  them  where  they  can  do  the  most  good,  and  advancing  only  the  most 
capable  individuals  can  be  increased.  Although  the  necessary  processes  are  difficult  and 
time  consuming,  the  benefits  can  be  enormous.  Only  when  selection,  assignment,  and 
promotion  are  recognized  as  major  opportunities  for  enhancing  the  organization,  rather 
than  as  chores  to  be  dealt  with  as  painlessly  as  possible,  will  an  organization  begin  to 
develop  its  full  potential. 

2.  Flexible,  adaptive  training  and  job  design.  Each  person's  style  of  learning  and 
working  is  different.  In  the  ideal  situation,  such  diversity  will  be  recognized  by  a  wide 
range  of  options  such  as  individualized  instruction,  flexitime,  job  enrichment,  and  a 
broader  choice  of  equipment  and  work  assignments.  Some  employees  prefer  less  rather 
than  more  job  enrichment--those  wishes  should  be  respected  (Lawler,  1974). 

3.  Individualized  incentive  system.  The  overriding  feature  of  an  individualized 
reward  system  is  recognition  that  productivity  almost  always  ranges  upward  from  a  ratio 
of  2:1  between  the  most  and  least  productive  workers.  With  explicit  recognition  of  this 
fact,  an  organization  is  likely  to  develop  an  incentive  program  that  will  entice  the  most 
able  workers  to  be  maximally  productive. 

Since  each  individual  has  his/her  own  hierarchy  of  values,  it  makes  sense  to  arrange  a 
reward  system  that  recognizes  individual  preferences.  Money  and  recognition  are 
obviously  common  rewards,  but  if  a  choice  of  rewards  is  provided,  the  worker  and  the 
organization  will  both  benefit.  In  a  comparison  of  f^jr  motivation  systems  at  each  of 
three  large  corporations,  Miller  (1981)  found  that  the  "pluralistic"  approach,  which 
recognized  individual  differences,  was  consistently  superior  to  the  others.  Longer 
vacations,  reserved  parking  spaces,  increased  autonomy,  and  other  prerequisites  are 
differentially  attractive  to  different  people  and  a  menu  of  incentives  should  be  offered  if 
at  all  possible.  The  military  environment  is  less  conducive  to  flexibility  in  this  matter 
than  its  civilian  counterpart,  but  effort  directed  toward  improving  flexibility  would  be 
well  spent. 

The  above  suggestions  for  taking  individuality  into  account  to  improve  organizations 
by  no  means  exhaust  the  possibilities.  For  example,  as  Coates  and  Kirby  (1982)  point  out, 
some  older  individuals  perform  at  a  higher  level  than  many  who  are  decades 
younger--thus  calling  into  question  the  wisdom  of  mandatory  retirement. 
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If  the  job  is  complex,  such  as  computer  programming,  electronics  troubleshooting, 
etc.,  differences  of  several  thousand  percent  are  frequently  found  (Williams  &  Rimland, 
1977).  A  good  example  of  individual  differences  in  programming  ability  was  reported  to 
the  authors  personally  by  a  rather  glum  research  assistant.  This  young  man  had  just 
returned  from  the  Southern  California  Programming  Competition,  where  he  and  several 
teammates  had  competed  against  14  other  teams  from  UCLA  and  elsewhere.  One  single 
member  of  the  UCLA  team  outperformed  each  of  the  other  teams,  each  of  which  had  at 
least  three  competitive  programmers! 

A  formal  demonstration  of  the  programming  phenomenon  appears  in  studies  of 
programmer  performance  and  training.  Meyer  (1965)  compared  two  methods  of  pro¬ 
grammer  training  and  found  that  although  the  means  of  the  two  groups  differed  by  only 
one  point  (49.5  errors  versus  50.5  errors),  the  individual  error  scores  ranged  from  30  to  70. 
Meyer  concluded  that  selecting  good  students  to  begin  with  was  perhaps  more  important 
than  improving  the  training  methods.  Grant  and  Sackman  (1966)  also  found  very  large 
differences  in  the  performance  of  programmers.  Examples  from  the  12  professional 
programmers  studied  are  given  below: 


Measure 

Poorest  Score 

Best  Score 

Ratio 

Debug  hours 

170 

6 

28:1 

CPU  time 

541 

50 

11:1 

Code  hours 

50 

2 

25:1 

Program  size 

6137 

1050 

6:1 

Run  time 

8.0 

0.6 

13:1 

The  Myth  of  Experience 

It  is  sometimes  thought  that  even  though  people  differ  initially  in  performance,  they 
tend  to  reach  the  same  level  as  they  gain  experience.  This  appears  to  be  untrue. 
Paterson  and  Darley  (1936;  cited  in  Maier,  1965),  for  example,  studied  three  groups  of 
typists  with  less  than  1  year,  1  to  5  years  ,  or  more  than  5  years  of  experience.  The  group 
with  less  than  one  year  of  experience  typed  an  average  of  39  words  per  minute,  compared 
to  42  words  per  minute  for  the  two  experienced  groups.  The  span  of  individual  ability  in 
every  group,  however,  was  immense:  speed  ranged  from  5  to  65  words  per  minute 
(corrected  for  errors).  Experience,  therefore,  did  not  affect  the  range  of  individual 
differences. 

Similarly,  Vineberg,  Sticht,  Taylor,  and  Cay  lor  (1971),  in  a  study  of  military 
technicians,  found  that  the  original  differences  in  the  aptitude  of  technicians  were 
reflected  in  the  range  of  performance  measured  5  years  later.  Tiffin  (1952),  in  discussing 
workers  in  industry,  suggested  that  training  tends  to  increase  individual  differences  in 
performance,  in  proportion  to  the  complexity  of  the  task  in  question. 

Discussion 


It  becomes  clear,  then,  that  the  individual  differences  in  performance  are  enormous. 
This  fact  is  especially  important  in  a  world  where  human  error  can  produce  catastrophic 
consequences.  As  Roger  Williams  (1978)  puts  it  in  his  book  Free  and  Unequal, 
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REFERENCE 


A  NEW  METHOD  FOR  COLLECTING  PROBLEM  SOLVING  STRATEGIES1 

Meryl  Baker 


Background 

One  of  the  most  essential  skills  in  Navy  maintenance  is  problem  solving.  Though  the 
specific  problems  vary  from  situation  to  situation,  there  appears  to  be  a  class  of 
individuals  who  are  effective  problem  solvers  in  almost  any  circumstance.  These  people 
appear  to  draw  upon  a  general,  content-free  set  of  strategies  which  facilitate  problem 
solving  ability.  The  purpose  of  this  research  was  to  explore  a  better  research  method  by 
which  to  study  generalizable  cognitive  strategies  so  that  trainable  components  can  be 
isolated.  These  might  eventually  be  taught  to  improve  the  problem  solving  ability  of  Navy 
maintenance  technicians. 

Most  recent  research  has  used  the  method  of  verbal  protocol  analysis  to  gain  an 
understanding  of  the  mental  processing  which  forms  the  basis  of  human  problem  solving 
behavior.  In  verbal  protocol  analysis,  an  experimenter  records  a  subject's  statements 
about  the  thought  processes  that  he  is  using.  The  researcher  usually  uses  a  tape  recorder, 
and  the  transcribed  tape  later  serves  as  the  basis  for  a  cognitive  model  of  the  problem 
solving  process. 

The  difficulty  with  verbal  protocol  analysis  is  that  subjects  may  not  or  cannot 
verbalize  all  of  their  thought  processes.  This  could  be  a  function  of  both  conscious  and 
subconscious  thought.  In  the  former  case,  subjects  may  provide  only  what  they  think  the 
researcher  wants  to  hear.  In  the  latter  case,  subjects  may  not  be  consciously  aware  of  all 
the  strategies  they  employ.  Some  mental  processing  may  occur  so  rapidly  and  sub¬ 
consciously  that  the  person  is  unable  to  recall  the  exact  problem  solving  process  even 
when  questioned  immediately  afterward.  This  phenomenon  appears  more  and  more 
frequently  as  we  move  into  the  high-speed  problem  solving  engaged  in  by  experts.  Experts 
think  so  rapidly  that  they  themselves  may  not  be  aware  of  the  strategies  they  use,  let 
alone  be  able  to  state  them  to  a  researcher  for  use  in  protocol  analysis.  Hence,  in  order 
to  more  reliably  investigate  cognitive  processing,  we  need  a  method  that  does  not  rely  on 
subject  recall. 

Approach 

The  present  study  tested  the  use  of  an  inexpensive  microcomputer  to  collect  subject 
strategy  data.  Puzzles  requiring  only  thought  and  logic  for  solution  were  selected.  The 
superiority  of  these  puzzles  over,  for  instance,  mathematical  problems,  lies  in  the  fact 
that  the  solutions  are  obtained  through  inherent  mental  prowess  rather  than  memory  of  a 
learned  set  of  rules  for  a  particular  content  area.  In  essence,  each  puzzle  epitomized  the 
problem  solving  process  because  its  solution  required  thought  processes  which  probably 
closely  resemble  those  employed  in  most  situations,  including,  for  example,  electronics 
troubleshooting. 

The  problems,  as  well  as  a  method  enabling  the  microcomputer  to  collect  subject 
solution  data,  were  programmed  onto  an  APPLE  II  PLUS  microcomputer.  The  programs 


*Work  performed  under  the  title  "Generalizable  Cognitive  Strategies"  (ZR000-01- 
042-06.017)  was  completed  in  FY84  and  transitioned  to  Exploratory  Development  task 
PE62763N,  RF63622801;  "Generic  Droblem  Solving." 


included  the  puzzle  text  and  a  puzzle  matrix,  which  was  to  be  used  by  subjects  as  a 
solution  board.  For  ease  of  manipulation  and  analysis,  the  data  were  transferred  from  an 
APPLE  II  PLUS  microcomputer  to  a  VAX  11/780  computer. 

Results 


The  rich  data  set  resulting  from  this  effort  will  be  used  in  future  research  to  form 
the  basis  of  a  cognitive  model  of  solution  strategies,  one  important  aspect  of  problem 
solving.  Though  this  model  and  the  instructional  strategies  to  be  derived  are  the  ultimate 
goal  of  the  overall  research  endeavor,  the  present  accomplishment  of  successfully 
collecting  subject  strategy  data  via  the  microcomputer  adds  significantly  to  the  field  of 
cognitive  research.  It  presents  a  cost  effective  means  for  collecting  and  analyzing 
subject  protocol  data.  In  future  research  efforts,  protocol  data  collected  via  this  method 
will  be  compared  with  verbal  protocol  data.  If  equal  or  better  quality  data  result  from 
the  new  method,  then  cost  and  time  savings,  and  increased  sophistication  in  subject 
comparisons  through  computer  data  manipulation,  would  support  this  method  as  an 
appealing  alternative  to  verbal  protocol  collection  and  analysis. 
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COGNITIVE  AND  EMOTIONAL  PROCESSING 


Barbara  McDonald 


Background 

Psychological  research  in  the  past  has  typically  separated  the  study  of  cognition  (e.g., 
learning,  memory,  skilled  performance)  from  the  study  of  emotion.  Although  scientist  and 
lay  person  alike  would  agree  that  emotion  plays  a  big  part  in  human  learning  and  task 
performance,  and  that  task  performance  plays  a  big  part  in  emotions,  psychologists  have 
not  studied  them  as  inseparable  processes.  Especially  in  this  age  of  high  technology, 
where  rational  problem  solving  and  pragmatic  decision  making  represent  the  popular  mode 
of  thinking,  emotion  has  often  only  seemed  to  complicate  the  important  issues  in 
cognition  (e.g.,  in  research  on  the  development  of  expert  systems  and  in  research  on 
reasoning  about  complex  systems).  As  a  result,  emotion  has  been  largely  left  out  of  the 
study  of  thinking  and  learning. 

In  spite  of  the  difficulties  of  incorporating  the  relationship  between  cognition  and 
emotion  into  a  scientific  analysis,  there  has  been  a  recent  upsurge  of  interest  within  a 
diverse  group  of  studies.  For  example,  studies  are  now  investigating  health  behaviors 
(people  who  express  their  fears  about  illness  report  much  less  physical  pain,  Leventhal, 
1982),  attitude  change  (people  have  a  difficult  time  changing  their  attitudes  even  when 
concrete  evidence  supports  a  different  position,  Zajonc,  1980),  and  mood-dependent  recall 
(people  remember  information  better  if  they  are  in  the  same  mood  during  learning  and 
recall,  Bower,  1981).  These  studies  all  suggest  the  importance  of  understanding  emotion 
to  the  understanding  of  learning  and  performance.  However,  up  to  this  point,  researchers 
have  not  looked  at  what  actually  happens  during  learning  and  performance  when  emotions 
are  involved.  Research  addressing  this  area  would  be  of  direct  relevance  to  the  study  of 
military  tactical  performance  because  military  tasks  are  often  performed  under 
frustrating,  anxiety-producing,  and  potentially  life-threatening  conditions. 

Objective 

The  purpose  of  the  present  research  is  to  investigate  the  mutual  influence  of  emotion 
and  cognitive  processing  on  learning  and  subsequent  performance.  The  objective  of  the 
first  year  was  to  develop  a  method  of  observation  for  the  study  of  the  influence  of 
emotion  and  cognition  on  learning  and  task  performance. 

Approach 

The  method  to  be  developed  had  to  meet  several  important  conditions:  First,  an 
experimental  scenario  was  needed  in  which  ongoing  behavior  could  be  monitored.  This 
suggested  a  naturalistic  approach  in  which  performance  data  and  emotional  reactions 
could  be  collected  simultaneously  during  task  performance.  At  the  same  time,  it  was 
critical  to  conduct  the  experiment  under  similar  conditions  for  each  subject.  It  was  also 
determined  that  a  task  should  be  observed  at  different  levels  of  difficulty  under  normal 
conditions  as  well  as  frustrating  (emotion-producing)  conditions.  Finally,  it  was  important 
that  the  task  be  realistic  and  absorbing  for  the  participants.  By  including  all  of  these 
conditions,  an  attempt  was  made  to  create  a  task  setting  in  which  the  key  elements  were 
adaptation  and  adjustment  to  task  demands  and  to  the  emotions  resulting  from  task 
performance. 


Results 


The  experimental  task  which  was  was  developed  involves  observation  of  experts  and 
novices  playing  a  computer-based  game  of  military  tactics.  The  eventual  participants  in 
the  study  will  be  instructors  and  students  at  the  Fleet  Combat  Training  Center  in  San 
Diego.  The  instructors  will  serve  as  experts  because  they  use  the  computer-based  game  in 
their  courses  as  well  as  teach  the  information  used  in  the  game;  the  students  will  serve  as 
novices. 

Each  subject  will  be  studied  individually  in  three  separate  conditions.  In  the  first 
session,  the  game  will  be  played  under  normal  conditions.  In  the  second  session,  the  game 
will  be  modified  to  include  frustrating  events  (buttons  will  not  work  as  usual  and  the 
timing  will  be  off  on  a  random  basis)  and  anxiety-producing  events  (false  information  and 
unexpected  events).  The  third  and  final  session  will  repeat  the  normal  condition.  For  all 
three  sessions,  the  games  will  be  played  at  basic,  intermediate,  and  advanced  levels  of 
difficulty. 

The  computer  will  collect  data  on  subject  skill  strategies  and  the  timing  of  strategy 
use.  All  three  sessions  will  be  tape-recorded.  The  role  of  the  experimenter  will  be  to 
encourage  the  player  to  talk  about  the  task  as  he  is  playing  the  game  and  to  develop  an 
atmosphere  in  which  the  player  feels  free  to  express  any  feelings  or  thoughts  he  might 
have.  The  experimenter  will  probe  the  player  for  strategy  information  and  engage  in 
conversation  if  the  player  wishes  to  discuss  something.  In  this  way,  observation  of 
emotional  reactions  can  be  done  unobtrusively  and  simultaneously  with  performance  of 
the  task.  Thus,  a  verbal  report  by  subjects  after  the  games  will  not  be  required. 

The  method  was  used  on  several  subjects  at  the  end  of  FY84  in  order  to  test  its 
feasibility.  The  task  and  the  method  of  observation  proved  to  be  very  successful.  The 
issues  of  primary  interest  were  (1)  whether  the  game  could  be  played  while  the  subject 
verbalized  thoughts  and  feelings,  (2)  whether  the  computer-based  game  was  realistic  and 
could  produce  identifiable  emotion,  and  (3)  whether  skilled  and  unskilled  task  performers 
could  experience  emotion  and  still  perform  the  task.  The  new  research  method  appears  to 
provide  the  needed  situation  and  information. 

In  FY85,  the  initial  study  using  the  new  method  will  be  completed.  After  tine  data 
analysis,  a  more  formal  study  will  be  proposed  in  which  more  sophisticated  measures  will 
be  used  (e.g.,  more  sensitive  indicators  of  emotional  reactions,  including  physiological 
measures^ 
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PROCEDURAL  TASKS:  WILL  TEACHING  THEORY  ENHANCE  RETENTION?1 

Paula  3.  Konoske 

3ohn  A.  Ellis 


Background 

Recent  surveys  have  found  that  most  military  tasks  are  procedural  (Van  Kekerix, 
1983;  Tarr,  1983;  Fredricks,  1981),  and  other  research  has  found  that  procedural  tasks  are 
the  most  important  type  for  Navy  mission  readiness  (Campbell,  O'Conner,  Sc  Peterson, 
1976).  That  is,  Navy  personnel  in  technical  ratings  must  maintain  high  levels  of 
procedural  skills  and  knowledge. 

Procedural  tasks  consist  of  an  ordered  sequence  of  steps  or  operations  performed  on  a 
single  object  or  in  a  specific  situation.  Procedural  tasks  involve  few  decisions  and  are 
generally  performed  the  same  way  each  time.  They  vary  in  the  amount  of  planning 
required,  the  number  of  steps  and  subprocedures,  the  amount  of  cueing  built  into  the  task, 
the  number  of  decision  points,  whether  or  not  the  order  of  the  steps  can  vary,  and  whether 
the  goals  of  the  task  are  internal  or  external  to  the  task,  system,  or  situation  (e.g., 
operation  vs.  maintenance). 

Unfortunately,  procedural  tasks  are  frequently  not  well  retained  (Schendel,  Shields,  Sc 
Katz,  1978;  Vineberg,  1975;  Ellis,  1980;  Hurlock  Sc  Montague,  1982).  One  possible  reason 
for  this  problem  is  that  most  instruction  for  procedural  tasks  is  "lean."  That  is, 
procedural  tasks  are  most  often  taught  as  a  linear  sequence  of  steps  with  a  single  top- 
level  goal.  The  instruction  typically  contains  little  explanation  ("theory")  about  the 
system  or  situation  and  how  the  steps  interrelate.  Students  are  usually  required  to 
memorize  procedural  information.  A  possible  solution  to  the  forgetting  problem  which 
typically  occurs  after  instruction  is  to  design  instruction  that  makes  the  task  more  easy  to 
remember.  This  project  investigated  the  effects  of  qualitative  explanations  or  elaborated 
instructions  (the  "theory"  behind  the  procedure)  on  learning,  performance,  and  long-term 
retention. 

Approach 

Qualitative  explanations  or  elaborated  instructions  are  what  Navy  instructors  and 
training  developers  are  talking  about  when  they  use  the  word  "theory."  Because  of  the 
controversy  in  Instructional  Systems  Development  (ISD)  and  Navy  technical  training  over 
how  much  theory  to  give  students,  it  is  important  to  determine  when  and  how  much  theory 
should  be  presented  for  specific  types  of  procedural  tasks.  Adding  theory  will  lengthen 
training,  but  more  theory  may  enhance  learning  and  retention.  The  scientific  rationale 
for  teaching  theory  is  that  qualitative  explanations  help  learners  build  mental  models 
(concrete  mental  representations)  of  systems  and  tasks.  Providing  students  with  supple¬ 
mental  explanations  sometimes  facilitates  learning  and  retention  of  procedural  and 
complex  rule-  and  principle-based  tasks  (Centner,  1980,  1981;  Smith  Sc  Goodman,  1982; 
Tourangeau  Sc  Sternberg,  1982;  Kieras,  1981;  Sturgis,  Ellis,  Sc  Wulfeck,  1981). 


'Some  of  the  work  undertaken  in  FY84  under  the  project  title  "Cognitive  Factors  in 
Learning  and  Retention"  (ZROOO-0 1-042-06.027)  will  be  used  for  Advanced  Development 
tasks  PE63720N;  Z 1772-002:  "Functional  Context  Training,"  and  PE62763N;  RF63522801: 
"Classroom  Instructional  Process." 
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Because  recent  reviews  of  the  scientific  literature  revealed  that  very  little  work  has 
been  done  on  procedural  tasks  and  that  various  authors  define  qualitative  explanations, 
analogies,  etc.,  and  procedural  tasks  in  different  ways  (Stevens  &  Steinberg,  1981;  Smith 
&  Goodman,  1982),  an  operationally  defined  taxonomy  of  types  of  qualitative  explanations 
was  developed.  Linear  types  (nonqualitative  presentations)  provide  the  student  with 
inventory  information  about  what  to  do  with  the  system  or  in  the  situation.  Structural 
explanations  provide  information  about  how  or  why  the  system  or  situation  is  constructed. 
Functional  explanations  provide  information  about  how  or  why  the  system  or  situation 
works.  After  the  types  of  explanations  were  identified,  the  research  relevant  to  each 
type  was  reviewed,  and  the  kinds  of  explanations  necessary  for  specific  procedural  tasks 
were  identified.  Because  procedural  tasks  differ  in  many  ways,  the  initial  part  of  this 
process  involved  identifying  types  of  procedural  tasks.  Four  different  "classes"  of  tasks 
were  identified:  (1)  operator  tasks,  (2)  maintenance/repair/assembly  tasks,  (3)  paper- 
based  tasks,  and  (4)  tasks  which  include  locating  information  or  objects.  Finally,  a  matrix, 
shown  in  Table  1,  combining  tasks  and  explanations  was  developed. 


Table  1 

Type  of  Procedural  Task  and  Type  of  Theory 
That  May  Improve  Retention 


Type  of 

Procedural  Task 

Type  of  Presentation 

Linear 

Structural  Theory 

How  Why 

Functional  Theory 

How  Why 

Operation 

* 

* 

* 

+ 

+ 

Maintenance 

Repair 

* 

+ 

+ 

+ 

+ 

Assembly 

* 

+ 

+ 

+ 

+ 

Paper-based 

Filing  Forms 

* 

+ 

Formatting  documents 

* 

+ 

+ 

+ 

Locating  information  or 

objects  * 

+ 

+ 

+ 

Note.  Qualitative  explanations  (theory)  can  be  direct  or  analogical. 
* 

Indicates  that  a  theory  can  be  applied  to  the  procedure. 


indicates  that  teaching  the  theory  behind  the  procedure  may  facilitate  performance/ 
retention. 


Table  1  indicates  our  hypothesis  for  each  type  of  task  and  explanation.  For  example, 
we  predict  that  the  "how"  type  of  structural  explanation  will  facilitate  retention  and 
performance  of  maintenance  procedures.  (Procedural  tasks  are  always  taught  by  a  linear 
presentation  whether  additional  explanations  are  presented  or  not.)  In  addition,  when 
functional  explanations  are  given,  they  sometimes  include  structural  information.  Thus, 
all  types  of  explanations  would  be  necessary  for  some  procedural  tasks  while  others  would 
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require  only  one  or  two  types.  For  example,  operating  the  brakes  of  a  car  does  not 
require  knowledge  of  how  the  brakes  are  structured,  but  it  does  require  linear  information 
about  the  effect  of  pushing  your  foot  on  the  pedal  as  the  cause  of  the  car  stopping. 
Repairing  the  brakes  of  a  car,  however,  requires  knowledge  about  how  the  brakes  are  put 
together  and  the  cause-effect  function  of  the  parts. 

Research  Plans 


We  proposed  to  conduct  two  experiments  in  FY85  to  test  the  hypotheses  generated  by 
Table  1  about  what  types  of  explanations  should  be  given  with  the  various  types  of 
procedural  tasks.  In  general,  the  experiments  will  examine  the  effects  providing  various 
types  of  explanations  on  the  learning,  performance,  and  retention  of  procedural  tasks.  We 
are  currently  planning  to  look  at  assembly  tasks.  We  may  also  vary  the  quality  of  the 
theory  presented  to  determine  the  characteristics  of  good  versus  poor  explanations.  The 
goal  for  FY85  is  to  complete  the  experiments  and  prepare  the  appropriate  journal 
articled)  and/or  technical  report(s). 
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ACQUIRING  SKILLS  FROM  WORK  EXPERIENCE1 

Robert  F.  Morrison 

Background 

This  research  sought  to  model  the  learning  from  work  experience  that  prepares 
people  for  successful  performance  in  their  career  roles.  Such  a  model  could  aid  the  design 
of  developmentally  oriented  career  patterns  that  optimize  learning.  Approaching  career 
development  from  the  perspective  of  experiential  learning  has  been  overlooked  in  theory, 
research,  and  application.  The  past  emphasis  has  been  on  formal  off-the-job  education 
and  training,  with  some  recent  emphasis  on  the  supervisor-subordinate  (mentor)  relation¬ 
ship.  Recent  studies  have  indicated  that  formal  education  and  training  account  for  only 
about  20  percent  of  work-related  learning,  with  the  remainder  accruing  from  experience 
(S.  Campbell,  personal  communication,  January  14,  1985). 

Approach 

The  research  contained  three  general  areas  of  concentration:  (1)  a  summary  of 
theory  and  technology  required  to  understand  and  apply  experiential  learning,  (2)  a  new 
theoretical  model,  and  (3)  the  collection  of  data  for  a  test  of  the  model  and  the 
development  of  measurement  techniques. 

Results 

Current  Theory  and  Technology 

A  review  of  the  literature  on  career  development  through  experiential  learning 
revealed  very  little  theory  or  application  of  direct  relevance.  A  survey  of  related 
constructs  and  technologies  identified  the  following  three  broad  areas  with  potential 
relevance: 

1.  Human  resources  management.  This  scientific  literature  on  career  paths  and 
patterns  proposes  use  of  subjective  job  titles  and  the  mathematical  modeling  of  historical 
data.  The  methods  do  not  consider  learning  as  a  central  theme  and  thus  do  not  allow  for 
the  design  of  new  developmentally  oriented  career  patterns  within  dynamic  work  environ¬ 
ments. 

2.  Learning  and  developmental  theory.  Many  concepts  from  the  huge  literature  on 
learning  theory  appear  to  apply  to  adult  development  during  their  careers.  It  is  necessary, 
however,  to  use  mega-extrapolation  from  the  time  frame  of  minutes  or  days  used  in  the 
theories  to  the  time  frame  of  many  years  that  comprise  a  career.  The  theories  that  are 
robust  enough  to  hold  up  under  time-frame  expansion  are  instrumental  learning  as 
interpreted  by  the  social  learning  theorists,  vicarious  learning,  learning  hierarchies,  and 
mastery  learning. 


*The  results  under  the  project  title  "Acquiring  Skill  from  Work  Experience"  (ZR000- 
01-042-08.023)  have  led  to  the  proposal  of  new  FY87  6.2  level  research  in  officer  career 
development.  That  research  would  look  at  both  the  optimum  length  of  operational  (sea), 
quasi-operational  (sea-related  shore),  and  nonoperational  (unrelated  shore)  tours  and  the 
sequences  of  tour  assignments  required  to  produce  effective  operational  and  nonopera¬ 
tional  commanding  officers.  Support  for  such  a  proposal  has  been  received  from  OP-13. 


3.  3ob/role  analysis.  A  great  deal  of  research  on  job  analysis  has  defined  jobs  in 
terms  of  specific  formal  duties,  tasks,  and  task  elements.  What  is  learned  from 
experience,  however,  is  frequently  complex  and  subtle,  so  most  of  these  approaches  are 
inadequate  for  the  study  of  experiential  learning.  The  term  "job"  must  be  reconceptual¬ 
ized  in  terms  of  the  various  roles  performed  at  work.  Many  of  these  informal  or 
interpersonal  roles  are  untapped  by  traditional  job  analyses,  require  a  great  deal  of 
learning  that  does  not  take  place  in  a  formal  classroom,  and  are  essential  to  successful 
career  development. 

Proposed  Model 

A  new  model  has  been  designed  during  this  research.  It  prescribes  the  constructs  that 
must  be  considered  to  enhance  the  development  of  work-related  knowledge  and  skills 
throughout  a  career's  work  experience.  The  model  provides  a  framework  for  a  technology 
that  can  integrate  experiential  learning  into  the  broader  context  of  adult  career 
development. 


Underlying  this  new  model  is  the  notion  that  career  experiences  can  be  planned  in 
logical  sequences  so  that  learning  is  maximized.  In  order  to  plan  an  experience  sequence, 
new  technology  for  analyzing  jobs  in  terms  of  roles  and  role-behaviors  must  be  devised. 
Career  paths  can  then  be  designed  so  that  individuals  have  maximum  opportunity  to  learn 
from  experience  the  behaviors  necessary  for  the  various  roles  encountered  throughout  a 
career  span.  Based  partly  on  the  concepts  from  learning  theory  mentioned  above,  it  is 
proposed  that  the  ideal  process  of  career  development  would  be  a  flexible  hierarchy  of 
work  experiences,  each  increasing  the  likelihood  of  success  at  the  next  level  and  building 
the  skills  needed  to  perform  well  in  a  set  of  target  career  roles.  Some  roles  are  unique  to 
entry-level  positions;  other  roles  become  more  complex  as  the  individual  transitions  from 
one  position  to  another.  Still  others  roles  are  prevalent  in  more  senior  positions. 

Pilot  Study 

An  exploratory  field  study  was  undertaken  to  enhance  the  theoretical  work,  support 
or  modify  the  model,  and  initiate  the  technology  required  to  operationalize  new  concepts. 
Interviews  with  Navy  officers  determined  some  of  the  various  roles  they  must  perform, 
the  knowledge  and  skills  needed  to  perform  them,  and  the  career  experiences  that  gave 
them  the  opportunities  to  learn  necessary  behaviors.  This  information  provided  some 
initial  support  for  the  experiential  career  learning  model.  Concomitantly,  some  factors 
that  should  be  considered  in  the  modification  or  revision  of  Navy  officer  career  patterns 
were  identified.  Use  of  these  factors  could  make  learning  faster,  more  thorough,  and 
more  focused  on  those  skills  that  are  required  for  present  and  future  job  performance. 


MODELS  OF  HUMAN  PERFORMANCE  WITH  APPLICATION  TO  DECISION  AIDING1 

Frank  L.  Greitzer 
Ramon  L.  Hershman 
C.  Rogers  Saxon 

Background 

The  nature  of  today's  threat  and  the  sophistication  of  modern  combat  systems  impose 
heavy  burdens  on  Navy  personnel.  While  innovations  in  automation  are  impressive,  they 
do  not  obviate  human  decisions;  rather,  operators  must  process  more  information  with  less 
time  and  error  than  ever  before.  Thus,  it  is  essential  that  designers  of  combat  systems 
and  decision  aids  consider  the  cognitive  skills  of  human  operators.  Data  are  required  on 
the  kinds  of  decision  strategies  they  invoke,  their  limitations  in  processing  information, 
and  the  effects  of  increased  workload  demands.  Computer  support  can  then  be  applied 
more  appropriately  to  enhance  the  decision-making  process. 

Approach 

Detecting  a  Change  in  Target  Location 

A  generic  surveillance  task  was  selected  to  serve  as  a  testbed  for  research  on 
modeling  cognitive  processes  and  evaluating  various  techniques  for  computer  aiding. 
Briefly,  the  task  is  as  follows:  The  decision  maker  must  decide  whether  a  target  is  at  a 
previous  fix  or  has  moved  to  a  new  location  offset  by  a  known  distance  and  a  random 
angle.  The  observer  views  a  sensor  display  that  shows  a  noisy  sample  of  N  reported 
locations  of  the  target.  The  noise  has  the  circular  normal  probability  density,  and  each 
datum  encodes  the  target's  reported  x,v  coordinates.  It  is  assumed  that  the  target  is 
stationary  at  the  time  the  data  are  observed  and  that  all  directions  of  movement  are 
equally  likely. 

In  the  most  straightforward  version  of  the  task,  the  data  appear  as  points  on  a  CRT. 
Figure  1  shows  two  typical  displays.  The  decision  maker  is  informed  that  the  target  was 
originally  fixed  at  the  center  of  the  circle  but  may  have  moved  R  units  to  the 
circumference.  Clearly,  the  target  has  moved  in  the  right-hand  display  but  not  in  the 
left-hand  display.  The  decision  maker  sees  only  one  such  display  and  must  infer  whether 
or  not  the  target  has  changed  location.  Of  course,  the  task  is  more  difficult  for  a  smaller 
sample  size  (N). 

Human  performance  data  and  theoretical  results  for  a  mathematically  optimal 
processor  have  been  described  earlier  (Hershman  &  Greitzer,  1982;  Greitzer  &  Hershman, 
1983).  In  this  FY84  effort,  decision  times  were  measured  for  alternative  display 
representations  and  varied  sample  sizes.  However,  attempts  to  quantify  the  observer's 
specific  information  processing  were  unproductive.  Decision  time  was  virtually  indepen¬ 
dent  of  N,  contradicting  the  notion  of  a  serial  processor  that  computes  the  optimal 


1  Results  from  the  project  titled  "  Models  of  Human  Performance  with  Application  to 
Decision  Aiding"  (ZROOO-0 1-042-09.022)  have  been  incorporated  into  Exploratory 
Development  Task  PE62757N:  SF57525601:  "Decision  Aiding  in  Combat  Systems." 


In  other  words,  the  people  who  probably  will  not  reenlist  appear  to  be  dissatisfied 
with  their  recent  duty  location  and  assignment  while  those  who  plan  to  reenlist  are  more 
concerned  with  holding  on  to  what  already  exists.  The  trends  are  similar  for  those  with 
more  years  of  service. 


In  sum,  the  demographic  and  incentive-preference  patterns  reveal  some  differentia¬ 
tion  between  low-  and  high-reenlistment-prone  personnel  within  each  years-of-service 
category.  As  years  of  service  increase,  demographics  become  less  discriminating  but  the 
pattern  of  incentive  preferences  persists.  Moreover,  in  all  cases,  the  high-reenlistment- 
prone  respondents  were  highly  sensitive  to  maintaining  retirement  benefits  and  promotion 
potential  at  no  lower  than  current  levels.  The  low-reenlistment-prone  respondents  also 
wanted  inducements  that  exceed  current  levels  in  terms  of  better  duty  assignments  or 
higher  economic  rewards,  the  latter  becoming  more  desired  as  years  of  service  increased. 

Optimal  Promotional  Themes 

From  the  lists  of  28  reasons  for  reenlisting  and  28  reasons  for  separating,  enlisted 
personnel  selected  their  most  salient  6  reasons  in  each  category.  In  the  analysis  of 
segment  differences,  six  very  popular  items  were  eliminated  and  an  optimal  package  of 
three  themes  was  selected.  For  example,  for  0-5  years  of  service,  the  three  themes  of 
"good  pay,"  "travel,"  and  "more  skill  training,"  reached  85  percent  of  the  respondents. 

Good  pay  and  travel  incentives  appeared  consistently  in  the  other  years-of-service 
categories,  but  the  groups  differed  on  the  third  theme,  skill  training.  Skill  training  was 
replaced  by  good  advancement  opportunities  (6-10  years  of  service),  duty  assignment  (11- 
14  years),  and  proud  to  wear  the  uniform  (15+  years).  Using  these  optimal  packages  rather 
than  simply  the  three  most  popular  themes  improved  audience  reach  by  a  maximum  of  5 
percent.  A  similar  analysis  was  conducted  with  the  separation  reasons. 

Optimal  Incentive  Menus 

Private  sector  employers  are  taking  action  to  cut  the  amount  they  spend  on  employee 
benefits  in  1985,  but  workers  may  find  that  these  fewer  dollars  stretch  farther.  One 
method  companies  use  to  help  employees  get  more  value  from  health  insurance,  life 
insurance,  and  other  benefits  are  the  new  "flexible"  benefits  plans.  According  to  an 
Associated  Press  report,  9  percent  of  the  companies  surveyed  already  had  flexible  benefit 
plans  and  37  percent  were  considering  them. 

One  objective  of  the  present  study  was  to  determine  an  optimal  menu  of  reenlistment 
benefits  that  might  comprise  a  type  of  flexible  benefit  plan  for  the  Navy.  By  "optimal 
menu"  is  meant  a  set  of  inducements  that  provides  the  maximum  net  increment  in 
likelihood  of  reenlistment  above  the  status  quo,  after  adjusting  for  the  cost  of  the 
inducement.  A  potential  reenlistee  can  select  a  single  inducement  (in  addition  to  the 
"standard"  package  of  inducements)  that  appears  best  for  him  or  her. 

There  are  two  criteria  for  selecting  an  optimal  menu.  First,  the  optimal  subset 
results  in  the  largest  net  gain  in  reenlistment  probability,  given  that  a  respondent  chooses 
the  inducement  that  most  increases  his  likelihood  of  reenlisting.  Second,  the  cost  of  the 
incentives  is  minimized.  Unfortunately,  it  is  very  difficult  to  place  a  dollar  value  on  the 
cost  of  each  incentive  and  the  trade-off  between  that  cost  and  its  negative  effect  on  the 
incremental  probability  of  reenlistment.  For  the  purposes  of  demonstrating  the  potential 
of  this  optimal  menu  technique,  three  Navy  officers  responsible  for  enlisted  retention, 
working  as  a  team,  rated  each  of  the  incentives  on  a  "difficulty-of-implementing"  scale. 


5.  Demographic  information  such  as  respondent  age  and  education. 

Analyses  and  Results 

Segmentation 

Prior  research  and  a  preliminary  analysis  led  to  the  selection  of  two  segmentation 
variables:  years  of  service  and  proneness  to  reenlist.  Four  years-of -service  categories 
were  created  based  on  the  selective  reenlistment  bonus  categories.  Proneness  to  reenlist 
was  defined  as  the  average  rating  the  respondent  gave  to  the  35  reenlistment  incentives. 
Within  years-of-service  intervals,  respondents  were  split  into  two  groups  depending  on 
whether  their  proneness  to  reenlist  was  above  or  below  the  median  for  their  group.  This 
segmentation  plan  led  to  eight  groups. 

Demographics 

Within  each  year-of-service  category,  a  discriminant  analysis  was  performed  to 
predict  high  versus  low  proneness  to  reenlist.  For  the  group  with  0-5  years  of  service,  the 
results  for  demographic  segmentation  show  that  low  proneness  to  reenlist  was  associated 
with  higher  pay  grades,  higher  AFQT  test  scores,  single  marital  status,  Caucasian  race, 
submarine  duty,  and  the  perception  that  similar  civilian  jobs  pay  more  than  the  Navy.  In 
other  words,  during  the  early  years  of  Navy  service,  the  bright,  single,  white  enlistee 
appears  to  be  less  prone  to  reenlist.  For  those  who  had  reenlisted  at  least  once, 
demographic  differences  between  personnel  with  low  and  high  proneness  to  reenlist  were 
less  discernable.  The  major  effect  of  demographic  variables  appears  to  occur  in  the  first 
5  years  of  service. 

Programs  and  Policies 

The  pattern  of  preferences  for  reenlistment  incentives  was  examined  by  combining 
the  35  reenlistment  incentives  and  the  seven  other  policies.  These  42  variables  were 
normalized  i'ound  each  respondent's  own  mean  and  standard  deviation  to  allow  analysis  of 
relative  preferences  for  the  42  policies.  A  discriminate  analysis  of  low  and  high  proneness 
to  reenlist  using  the  42  variables  as  predictors  was  performed  separately  by  years-of- 
service  categories. 

The  major  distinctions  between  the  low  and  high  reenlistment  groups  for  personnel 
with  0-5  years  of  service  were  as  follows: 

Low  proneness  to  reenlist  was  associated  with  high  preferences  for  guaranteed: 

1.  First  choice  of  duty  location. 

2.  First  choice  of  duty  assignment. 

3.  Permanent  duty  location  of  up  to  5  years. 

High  proneness  to  reenlist  was  associated  with  greater  sensitivity  to: 

1.  Lack  of  a  reenlistment  bonus. 

2.  Delay  of  retirement  to  age  60. 

3.  Increased  age  for  retirement  eligibility. 

4.  Reduced  chances  of  promotion. 

5.  Availability  of  a  firm  civilian  job. 
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APPLICATION  OF  MARKET  SEGMENTATION  TO  NAVY  RETENTION  ISSUES 


Susan  Hearold 


Background 

Market  segmentation  is  concerned  with  individual  or  group  differences  in  response  to 
products.  It  provides  guidelines  for  a  firm's  marketing  strategy  and  resource  allocation 
among  markets  and  products.  The  presumption  is  that  if  consumer  response  differences 
exist,  can  be  identified,  and  are  reasonably  stable  over  time,  the  firm  can  increase  its 
sales  and  profits  beyond  those  obtained  by  assuming  market  homogeneity. 

The  application  of  market  segmentation  is  not  limited  to  consumer  goods.  In  keeping 
with  marketing's  broadening  scope  and  the  position  that  many  forms  of  human  exchange 
belong  within  the  domain  of  marketing,  social  products  increasingly  appear  as  foci  of 
marketing  thought.  Market  segmentation  has  been  applied  to  such  diverse  fields  as  the 
development  of  nutrition  programs  in  India,  the  marketing  of  zoos,  and  the  specification 
of  strategies  for  economic  development  in  Columbia. 


Approach 

This  study  used  several  market  segmentation  procedures  to  analyze  the  responses  of 
Navy  enlisted  personnel  to  reenlistment  incentives.  The  primary  dependent  measure, 
intention  to  reenlist,  is  analogous  to  a  preferred  consumer  product  measure,  intention  to 
purchase. 

Sample 

The  survey  sample  was  a  representative  group  of  all  Navy  nonreserve  and  rated 
enlisted  personnel  whose  reenlistment  decision  was  due  within  the  next  12  months.  The 
population  was  stratified  by  rating  and  number  of  enlistments,  and  a  proportional  sample 
was  selected.  Surveys  were  sent  to  2825  personnel.  Of  those,  1309  usable  questionnaires 
were  returned  (a  46%  return  rate). 

Survey 

The  questions  were  presented  in  seven  sections,  five  of  which  were  used  in  this 
market  segmentation  study: 

1.  Present  reenlistment  intention  rated  on  a  0-10  probability  scale  indicating 
current  likelihood  of  reenlisting. 

2.  Thirty-five  reenlistment  incentives  (e.g.,  pay,  shore  duty),  each  rated  on  the 
same  0-10  point  probability  scale  indicating  likelihood  of  reenlisting  if  just  this  one 
change  were  made  in  current  benefits. 

3.  Seven  reenlistment  policies  that  were  not  necessarily  reenlistment  incentives 
(e.g.,  retirement  possible  after  10  years),  each  rated  on  the  same  0-10  point  scale. 

4.  A  list  of  28  reasons  for  reenlisting  and  28  reasons  for  separating,  from  which  the 
respondents  were  asked  to  pick  6  from  each  group. 
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This  independent  exploratory  development  project  was  formally  initiated 
on  29  March  1983.  The  conceptual  and  developmental  work  were  completed  through  the 
end  of  FY84.  The  basic  real-time  simulation  model  has  been  designed  and 
implemented  on  a  TERAK,  It  provides  for  player  interaction  and  responds  to 
commands  to  employ  ship  defense  measures.  The  player  query,  response  handling,  and 
performance  evaluation  processes  have  been  designed  and  implemented,  and  the  platform 
and  weapon  system  characteristic  data  bases  have  been  completed.  The  development  of 
the  simulation  model  and  the  query/response  features  has  been  completed  and  the  threat 
and  ownship  data  bases  have  been  filled.  Formative  evaluation  has  been  conducted  at 
NPRDC  and  at  the  Fleet  Combat  Training  Center,  Pacific  (FCTCP). 

The  tactical  decision  trainer  project  (PE63720N)  initiated  in  FY85,  aimed  at 
providing  training  at  the  unit  level  in  multiship,  multithreat  environments,  is  building  upon 
the  training  concepts  developed  in  the  single-ship,  single-threat  (AAW)  training  situations 
provided  for  in  the  self-defense  tactical  trainer.  Additionally,  the  SDTT  system  will  be 
implemented  at  FCTCP  to  supplement  TAO  training  in  a  manner  that  has  proven 
interesting  and  enjoyable  for  the  students. 


Table  1 


Features  of  the  SDTT  Player-Computer  Interactions 


Program  Features 


General: 

1.  Written  in  UCSD  Pascal  for  the  TERAK. 

2.  The  U.S.  Navy  surface  combatant  is  selectable  by  the  player  or  can  be  randomly 
selected  by  the  program  (all  current/projected  ship  classes  are  modeled). 

3.  The  threat  platform  is  selectable  by  the  player  or  by  the  program  (all  ASCM  threats 
in  the  TAO  matrix  are  modeled). 

Before  the  Real-time  Wargame: 

1.  Queries  the  player  on  data  associated  with  the  selected  U.S.  Navy  ship  or  threat,  if 
known  (weapons  carried,  emitters  associated,  launch  ranges,  weapon  characteristics, 
etc.). 

2.  Records  player  responses. 

3.  Provides  limited  tutoring  for  incorrect  responses. 

4.  Simulates  the  operation  of  threat  emitters  in  prelaunch  or  launch  condition. 

During  the  Wargame: 

1.  Generates  up  to  three  different  ASCMs  aimed  at  the  U.S.  Navy  ship. 

2.  Simulates  ASCM  flight  profiles  and  emitter  employment  during  flight. 

3.  Permits  the  player  to  enter  commands  to  defend  his  ship,  including  the  employment 
of  surface-to-air  missiles  (SAMs),  BPDMS,  CIWS,  chaff,  and  SLQ-32  deception  and 
jamming,  if  the  selected  ship  is  so  equipped. 

4.  Graphically  presents  engagements,  using  symbology  and  track  numbers  similar  to  the 
Navy  Tactical  Data  System  (NTDS). 

5.  Provides  continuously  updated  alphanumeric  data  on  signal,  target,  weapon,  and 
command  status. 

6.  Computes  outcomes  of  Hardkill  weapon  engagements  with  ASCMs. 

7.  Computes  effectiveness  of  Softkill  (electronic  countermeasures)  system  employ¬ 
ment. 

8.  Displays  engagement  results  graphically  and  alphanumerically. 

Af  ter  the  Wargame: 

1.  Evaluates  the  defensive  measures  taken  by  the  player  (calculates  the  cumulative 
effects  of  the  player's  actions  on  the  probabilities  that  the  ASCMs  hit  the  ship). 

2.  Scores  the  player  on  his  responses  to  queries  and  the  tactical  employment  options 
used  in  the  wargame. 


THE  SELF-DEFENSE  TACTICAL  TRAINER1 

David  W.  Anderson 


Background 

The  tactical  proficiency  of  naval  officers  has  been  an  issue  of  major  concern  in 
OPNAV  for  many  years.  Present  tactical  training  includes  courses  and  team  training  at 
Fleet  Combat  and  ASW  Training  Centers  and  infrequent  at-sea  or  inport  multiship 
exercises.  Tactical  action  officer  (TAO)  courses  are  extraordinarily  rigorous  and  require 
memorization  of  a  tremendous  amount  of  data  on  platform  and  weapon  system  character¬ 
istics  and  tactics.  These  include  ownship  sensor  and  weapon  characteristics,  threat 
platform  sensor  and  weapon  characteristics,  antiship  cruise  missile  (ASCM)  employment 
tactics,  and  employment  of  "Hardkill"  (target  destroyed)  and  "Softkill"  (target  decoyed  or 
jammed)  ASCM  countermeasure  systems.  There  are  few  means  to  refresh  and  reinforce 
the  knowledge  after  the  TAO  returns  to  his  ship.  Consequently,  there  has  been  significant 
high-level  interest  in  revitalizing  tactical  training  through  microcomputers  and  computer- 
assisted  training  (CAI)  techniques,  both  to  improve  proficiency  and  to  move  more  training 
aboard  ship. 

Objective 

The  objective  was  to  design  and  develop  a  prototype  microcomputer-based  self- 
defense  tactical  trainer  (SDTT)  to  exercise  and  test  an  individual  officer's  threat 
assessment  capabilities,  tactical  knowledge,  and  reactions.  The  exercise  and  test  were  to 
be  performed  in  real-time  or  near  real-time  against  a  computer -generated  ASCM  threat. 
Such  a  system  could  be  used  for  initial  training  in  the  schoolhouse  and,  because 
microcomputers  are  portable,  for  refresher  training  onboard  ships  or  at  Fleet  Training 
Groups.  Primarily,  this  software  development  was  undertaken  in  anticipation  of  transition 
to  an  advanced  development  effort  in  tactical  decision  training. 

Approach 

An  interactive  CAI/computer  wargame  simulation  program  was  designed  to  run  in 
UCSD  Pascal  on  the  TERAK  microcomputer.  The  program  consists  of  procedures  to 
select  a  USN  platform  and  ASCM;  draw  and  refresh  the  screen  and  symbols;  generate 
appropriate  electronic  signals;  handle  the  missile  and  ownship  graphics;  provide  alpha¬ 
numeric  data  on  signal,  target,  and  weapon  status:  and  compute  outcomes  of  A  AW  missile, 
close-in  weapon  system  (CIWS),  and  basic  point  defense  missile  system  (BPDMS)  engage¬ 
ments  with  ASCMs.  The  program  also  computes  effectiveness  of  electronic  counter¬ 
measures  such  as  chaff  and  SLQ-32  jamming  and  deception.  CAI  techniques  are  used  to 
query  the  player  interactively  on  ownship  and  threat  characteristics  and  tactical  aspects 
of  the  scenario  presented,  to  score  responses,  and  to  provide  feedback  on  the  simulated 
tactical  measures  the  player  uses  in  defending  his  ship.  The  system  is  user-friendly 
enough  to  allow  the  officer  with  very  little  computer  experience  to  set  up  and  "play." 

Table  I  describes  features  of  the  SDTT  and  illustrates  the  approach  and  player- 
computer  interactions. 


*Work  under  the  project  title  "Self-Defense  Tactical  Trainer"  (ZF66-512-001.008)  has 
been  transitioned  to  Advanced  Development  task  PE63720N,  Z1772:  Tactical  Decision 
Training. 
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Results 


An  empirical  model  was  developed  in  which  the  transportation  section  of  the  network 
was  developed  to  handle  policies  at  the  operational  level  and  the  transshipment  section  of 
the  network  was  developed  to  handle  policies  at  the  manpower  planning  level.  The  special 
contribution  of  this  effort  is  the  development  of  a  dynamic  process  to  generate  and  to 
integrate  all  possible  manpower  goals  into  the  transshipment  section  of  the  model.  The 
policy  objectives  are  optimized  preemptively  one  after  another  by  using  the  sequential 
elimination  method  or  the  composite  objective  function.  This  work  supports  the 
theoretical  development  of  the  individual  research  (see  Liang,  1984). 

Applications 

The  methodology  was  applied  to  a  6.3  project  (personnel  assignment  system)  to 
develop  an  automated  personnel  allocation  and  assignment  system  for  the  nonskilled 
enlisted  personnel.  It  was  the  first  successful  attempt  in  developing  an  automated  policy 
planning  and  operation  system  for  personnel  assignment  in  the  Navy.  The  Navy's 
assignment  and  allocation  offices  are  in  the  process  of  implementing  the  new  system. 
This  system  will  be  expanded  to  handle  skilled  enlisted  personnel  as  well.  In  addition,  the 
work  is  being  applied  to  other  6.2  and  6.3  projects  related  to  career  management  planning, 
personnel  classification,  and  sea/shore  rotation. 

Research  Potential 


Although  the  multiple  criterion  optimization  technique  developed  from  this  effort  is 
sufficient  for  existing  projects,  more  research  is  needed  to  solve  more  complex  problems. 
In  particular,  additional  research  is  needed  to  (1)  improve  measurement  of  the  trade-offs 
among  policies,  (2)  increase  the  flexibility  of  the  technique  to  incorporate  more  policy 
objectives  and  constraints,  and  (3)  evaluate  the  impact  of  various  policies. 


MULTIPLE  CRITERION  OPTIMIZATION  TECHNIQUES1 

Timothy  T.  Liang 


Background 

Accession,  promotion,  rotation,  and  attrition  frequently  require  the  Navy  to  redistri¬ 
bute  its  personnel  to  major  units  and  reassign  individuals  to  jobs.  In  matching  people  to 
jobs,  the  decision  makers  consider  numerous  rules  and  regulations  on  people-jobs  eligibil¬ 
ity.  A  person  might  be  eligible  for  many  jobs  and  a  job  might  be  suitable  for  many  people. 
However,  a  person  can  only  be  assigned  to  one  job  and  the  Navy  assigns  people  according 
to  current  policies.  There  are  many  microlevel  policies  involved:  For  example,  the  Navy 
has  always  tried  to  match  personnel  skills  with  job  requirements  to  maximize  resource 
utilization.  Navy  policy  also  tries  to  minimize  personnel  moves  from  one  location  to 
another,  thus  saving  moving  expenses.  The  Navy  also  tries  to  assign  personnel  to  the 
location  of  their  choice,  when  possible,  to  satisfy  individuals  and  to  increase  retention. 
With  these  multiple  and  often  conflicting  policies  included  in  the  decision  process,  it  is 
unlikely  that  the  decision  makers  can  satisfy  all  policy  goals  and  requirements  without  a 
systems  approach. 

Recently,  the  Navy  has  been  increasingly  concerned  about  manpower  planning,  and 
macrolevel  policies  have  been  introduced  to  improve  aggregate  manpower  distribution. 
Manning  (percentage  of  positions  filled)  is  frequently  used  to  measure  manpower  balance. 
Some  policies  are  directed  to  balanced  manning  between  the  Pacific  Fleet  and  the 
Atlantic  Fleet;  other  policies  are  developed  to  create  a  manning  ratio  for  sea  and  shore 
duty.  The  decision  makers  assigning  people  to  jobs  within  the  context  of  these  macrogoals 
are  already  so  burdened  by  policies  at  the  microlevel  that  making  optimal  decisions  is 
difficult.  Research  is  needed  to  develop  a  large-scale  model  to  study  multiple-objective 
decisions. 

Multiple  criteria  models  have  been  a  subject  of  accelerated  interest  during  the  past 
decade.  The  major  efforts  focus  on  algorithms  to  estimate  the  numerical  weights  of 
multiple  policies.  Conventional  mathematical  programming  approaches,  however,  are  not 
capable  of  efficiently  solving  large  scale  problems  such  as  the  Navy's  personnel  distribu¬ 
tion  problems,  particularly  for  those  problems  that  require  integer  solutions.  Remarkable 
technological  breakthroughs  in  network  optimization  during  the  past  few  years  have 
enabled  routine  and  efficient  solutions  to  large  and  complex  planning  problems.  They 
provide  the  opportunity  to  develop  a  network  formulation  for  multiple-objective  problems. 
The  Individual  Research  effort  of  this  project  in  FY83  developed  a  technique  that 
demonstrated  the  feasibility  and  capability  of  developing  a  network  approach  to  study  the 
large  scale  multiple-criterion  optimization  problems. 

Objective 

The  purpose  of  the  current  IED  effort  is  to  use  the  technique  developed  under 
previous  Independent  Research  and  the  data  from  the  Navy's  personnel  assignment 
problem  to  derive  an  empirical  model  to  help  decision  makers  assign  personnel  to  jobs  in 
accordance  with  multiple  and  conflicting  policies. 


*The  results  of  work  under  the  project  title  "Multiple  Criterion  Optimization 
Techniques"  (ZF66-51 1.013)  have  influenced  work  in  three  FY85-funded  efforts:  (a) 
PE63707N,  Z1770:  Personnel  Assignment  System,  (b)  PE62763N,  RF63-521-004-031: 
Career  Management  Planning,  and  (c)  PE99000N,  C0073-002.02:  CLASP  System 
Development. 
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INDEPENDENT  EXPLORATORY  DEVELOPMENT  (IED)  PROGRAM 

The  Independent  Exploratory  Development  (IED)  Program  at  Navy  Personnel  Re¬ 
search  and  Development  Center  was  initiated  in  FY76.  Work  in  this  funding  category  is 
intended  to  develop  or  evaluate  basic  research  with  a  focus  on  specific  military  problem 
areas.  IED  work  units  for  FY84  and  FY85  are  listed  in  Table  2.  Several  of  the  FY84 
research  efforts  will  be  transitioned  to  FY85  projects  in  the  areas  of  tactical  training  (see 
Anderson,  p.  38),  personnel  assignment  and  career  planning  (see  Liang,  p.  36),  and 
authoring  instructional  materials  (see  Chang,  p.  46).  Selected  work  conducted  within 
these  units  during  FY84  is  described  in  the  remainder  of  this  section. 


Table  2 

Independent  Exploratory  Development  Work  Units  for  FY84  and  FY85 


62766N 

Project  Title 

Principal 

Investigator 

Funding  in  $K 
FY84  FY85 

•  .  '« 

ZF66-51 1.013 

Multiple  Criterion  Optimization 

Techniques 

Dr.  T.  Liang 

AV:  933-2371 

Code  61 

40 

15 

i.. .  ^ 

ZF66-512.008 

Self-defense  Tactical  Trainer 

CDR  D.  Anderson 
AV:  933-7100 

Code  05 

26 

0 

' 

.009 

Application  of  Market  Segmentation 

Dr.  S.  Hearold 

27 

0 

r*  - 

to  Navy  Retention  Issuesa 

AV:  933-2081 

Code  71 

.012 

Alternative  Military  Manpower 

Dr.  3.  Borack 

50 

0 

Supply  Methodologies 

AV:  933-2371 

Code  61 

Ik.  « 

.014 

Development  of  Computer-based 

Writing  Aids 

Dr.  F.  Chang 

AV:  933-6434 

Code  51 

15 

0 

.015 

Interfaces  for  Combat  Decision  Support 

Dr.  F.  Greitzer 

AV:  933-2081 

Code  71 

0 

90 

.016 

Impact  of  Statistical  Process  Control 

Dr.  L.  Broedling 
AV:  933-6620 

Code  52 

0 

10 

r  “  1 

* 

.017 

Development  of  Graphic  Design  Aids 

L.  Weitzman 

AV:  933-6242 

Code  501 

^  1 

oo  |o 

JO 

165 

v> 

*  o 

•  s 

■ 
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Completed  in  FY84. 
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that  target  that  will  reach  ownship  the  soonest."  Note  that  this  strategy  compels  the 
operator  to  integrate  both  target  range  and  speed  in  establishing  firing  priorities.  An 
alternative,  range  strategy,  is  simply  to  "Always  fire  at  the  closest  target."  Since  the 
range  strategy  requires  only  one  attribute  for  its  implementation,  its  processing  rate 
should  be  higher. 

Our  experiment  manipulated  two  factors:  strategy  instruction  and  workload.  In  one 
condition,  an  experienced  air  intercept  controller  was  asked  to  prosecute  targets  as  usual. 
In  another  condition,  he  was  explicitly  instructed  to  use  the  vulnerability  strategy. 
Workload  was  varied  by  using  two  target  arrival  rates. 

A  mathematical  model  based  on  the  target  rankings  for  range  and  vulnerability  was 
developed  and  tested.  Using  only  two  parameters,  the  model  makes  predictions  for  the 
probability  that  the  jth-ranked  of  k  targets  (based  on  either  presumed  strategy)  will  be  the 
one  selected  for  firing.  It  was  found  that  the  operator's  normal  processing  mode  was  a 
range  strategy,  although  he  could  adopt  the  more  complex  vulnerability  strategy  when  so 
instructed.  However,  despite  its  defined  empirical  optimality,  the  vulnerability  strategy 
led  to  poorer  performance  (increasing  the  threat  to  ownship),  presumably  due  to  its 
greater  processing  burden.  There  was  no  evidence  of  a  shift  in  strategy  due  to  workload; 
rather,  it  seems  that  increased  workload  limited  the  depth  of  processing  to  which  the 
operative  strategy  was  pursued. 

Another  experiment  briefly  tested  various  aids.  An  operator  performed  either 
unaided  or  was  assisted  by  the  computer-driven  display  that  showed:  (Aid  1)  the  single 
most  threatening  target;  (Aid  2)  the  most  threatening  fast,  medium,  and  slow  target 
choices;  or  (Aid  3)  the  three  best  choices  regardless  of  speed.  It  was  presumed  that  Aid  1 
would  be  most  effective;  the  other  aids  were  expected  to  assist  the  operator  but  lead  to 
an  increased  processing  demand.  The  result,  however,  showed  that  compared  with  the 
unaided  condition,  the  aids  reduced  ownship  vulnerability  by  31,  58,  and  78  percent 
respectively.  Thus  Aid  3  was  best,  permitting  the  operator  to  prosecute  targets  at  a  high 
rate  with  minimal  loss  in  decision  quality.  Although  Aid  1  had  been  designed  for  optimal 
target  selection,  it  promoted  dependency  on  the  computer  and  reduced  the  output  rate. 

Conclusions 

1.  The  operator’s  adoption  of  a  simple  range  strategy  is  interpreted  as  an  optimal 
use  of  nonoptimal  resources.  It  is  thus  a  good  example  of  well-advised  coping  with 
formidable  processing  demands  and  confirms  in  a  dynamic  decision  task  the  earlier 
findings  of  Rigney  and  DeBow  (1967). 

2.  Strategies  used  by  "experts"  are  not  necessarily  optimal.  They  may  not  be  the 
best  guidelines  for  automated  algorithms. 

3.  Detailed  data  collection  and  rather  simple  mathematical  models  (based  only  on 
rank-order  information)  are  useful  in  making  inferences  about  an  operator's  threat 
assessment  strategies. 

4.  Decision  aids  that  promote  the  user's  dependency  on  the  computer  should  be 
avoided.  User-computer  collaboration  is  a  more  effective  approach,  but  it  requires 
increased  attention  to  human  skills  and  limitations. 
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Figure  1.  Two  typical  displays  with  N=12.  In  the  left-hand  display, 
the  target  has  not  moved,  and  the  reported  positions 
cluster  around  its  original  fix  at  the  center  of  the  circle. 
In  the  right-hand  display,  the  target  has  moved  to  the 
circle's  circumference  at  an  angle  of  approximately  45 
degrees. 


decision  statistic—namely,  the  distance  from  the  origin  to  the  centroid  (or  center  of 
gravity)  of  the  displayed  data.  Efforts  to  aid  the  observer  by  displaying  the  centroid  or 
the  criterion  distance  were  ineffective.  It  was  concluded  that  an  observer's  solution  is  by 
and  large  perceptual,  rather  than  cognitive;  that  is,  figural  pattern  processing  is  invoked 
rather  than  computational  algorithms.  Accordingly,  the  surveillance  task  was  deemed 
unsuitable  for  the  planned  investigations  and  a  different  context  for  the  research  was 
selected. 

Threat  Assessment  Strategies 


This  research  sought  to  infer  human  information  processing  strategies  from  observed 
firing  priorities  in  simulated  antiair  warfare  (A AW)  engagements.  The  operator's  task  was 
to  defend  "ownship"  by  launching  missiles  against  a  raid  of  incoming  targets  that  were 
equally  lethal  and  approached  at  one  of  three  speeds  from  various  bearings.  Figure  2 
shows  a  snapshot  of  the  simulation  in  progress. 

The  research  issues  were:  (1)  What  is  the  operator's  normal  processing  mode?  (2) 
Does  an  explicit  processing  instruction  yield  evidence  that  the  instructed  strategy  is 
indeed  invoked?  (3)  Does  increased  workload  produce  an  identifiable  shift  in  strategy?  (4) 
What  are  the  implications  for  the  design  of  decision  aids? 

Although  interception  of  all  targets  at  maximum  range  is  the  ideal  goal,  limits  in 
human  processing  made  it  more  appropriate  to  seek  to  minimize  ownship's  vulnerability,2 
or  total  exposure  to  threat.  Thus,  a  vulnerability  strategy  was  defined  as  ''Always  fire  at 


2 Let  the  momentary  vulnerability  induced  by  a  target  be  equal  to  1/t  where  t  is  the 
time  remaining  until  the  target  would  penetrate  and  score  a  hit  against  ownship.  The 
total  vulnerability  incurred  in  an  entire  AAW  engagement  is  then  taken  as  the  sum  of  the 
momentary  vulnerabilities,  accumulated  over  all  targets  and  all  updates. 
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Procedurally,  the  increased  likelihood  of  reenlisting  was  debited  by  the  difficulty-of- 
implementing  scale  rating. 

Eighteen  incentives  thought  to  be  the  most  amenable  to  change  by  the  Navy  were 
used  in  this  analysis  for  each  segment.  The  optimal  menu  of  five  incentives  that  was 
identified  varied  by  segment,  but  several  incentives  appeared  frequently.  The  most 
popular  incentives  (adjusted  for  difficulty  of  implementing)  across  all  eight  segments 
were: 

1.  Guaranteed  one  of  their  top  three  choices  of  duty  location. 

2.  Free  dental  coverage  for  dependents. 

3.  Permanent  duty  location  for  up  to  5  years. 

4.  Indefinite  reenlistment  contract. 

Each  of  these  appeared  at  least  six  times  across  the  eight  segments  and  showed  high  net 
gain  across  the  sample.  In  particular,  the  incentive  to  allow  permanent  duty  location  for 
up  to  five  years  provides  an  extremely  good  return  even  though  its  difficulty  of 
implementation  rating  is  quite  high. 

Summary 

After  various  analyses  probing  numerous  different  reenlistment  market  segments,  the 
most  consistent  finding  is  that  the  influential  incentives  are  strong  across  all  groups.  The 
pattern  of  incentive  effect  on  increasing  or  decreasing  reenlistment  intention  is  basically 
the  same  for  identifiable  groups.  This  effort  included  two  unique  analyses:  (1) 
measurement  of  the  maximal  reach  for  sets  of  themes  to  promote  reenlistment,  and  (2) 
identification  of  optimal  reenlistment  incentive  menus.  Market  segmentation  techniques 
are  appropriate  and  efficient  for  application  to  retention  issues. 
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ALTERNATIVE  MILITARY  MANPOWER  SUPPLY  METHODOLOGIES 


Jules  I.  Borack 


Background 

Although  military  manpower  specialists  have  used  the  phrases  "supply  of  manpower 
for  military  service"  or  "military  personnel  supply"  with  increasing  frequency,  there  is  no 
commonly  accepted  definition  of  the  term  "military  manpower  supply."  Rather,  the  term 
supply  appears  to  symbolize  the  method  used  by  an  individual  planner  or  researcher, 
rather  than  an  underlying  phenomenon.  Because  of  the  bewildering  assortment  of 
definitions  and  projections,  a  prospectus  that  combines  various  manpower  supply  methods 
into  a  logical  framework  is  needed. 

Objectives  and  Approach 

The  primary  objectives  of  this  effort  were  to  analyze  the  principal  methods  used  by 
military  manpower  supply  specialists  and  to  suggest  a  prospectus  for  combining  them  into 
a  logical  framework.  Three  distinct  methods  used  by  manpower  supply  researchers  were 
analyzed:  (1)  econometric  model  development,  (2)  surveys  of  interest/intentions  to  join 
the  military,  and  (3)  demographic  analyses. 


Results 


The  definitions  of  supply  inherent  in  the  approaches  addressed  are  quite  dissimilar. 
Econometric  models  define  supply  as  the  expected  value  of  "high-quality"  contracts  or 
accessions;  surveys  define  supply  as  the  number  and  composition  of  individuals  who  are 
planning  (or  are  interested  in)  a  military  career;  and  demographic  approaches  define 
supply  as  the  number  and  composition  of  individuals  who  are  able  to  fulfill  military 
entrance  criteria. 

Each  method  possess  several  strengths  and  weaknesses.  Econometric  models  are  easy 
to  implement  and  define  supply  in  a  directly  usable  fashion.  However,  they  are  unable  to 
incorporate  variables  that  are  not  directly  quantifiable  and  models  cannot  be  easily 
developed  for  demand-constrained  or  nontraditional  populations.  Conversely,  manpower 
supply  surveys  do  not  define  supply  in  a  directly  implementable  manner  but  are  very 
useful  in  measuring  the  relative  size  of  a  supply  pool.  In  addition,  these  surveys 
investigate  the  entire  spectrum  of  the  nation’s  youth--not  just  those  youths  who  have 
chosen  to  join  the  military.  Finally,  demographic  approaches  are  useful  in  providing  a  tool 
for  assessing  the  impact  of  alterations  in  accession  standards.  Demographic  techniques, 
however,  do  not  by  themselves  aid  in  the  assessment  of  key  policies  that  would  permit  the 
military  to  balance  supply  and  demand  efficiently. 

Conclusions  and  Recommendations 

These  findings  indicate  that  no  single  method  provides  al!  the  information  necessary 
for  a  full  understanding  of  the  accessioning  process.  A  "good"  supply  model  should  use  the 
best  demographic  techniques  to  establish  accurate  bottom-line  populations  and  combine 
them  with  interest/intent  data  obtained  through  surveys  and  relevant  economic  pre¬ 
dictors.  A  logical  framework  that  integrates  these  diverse  approaches  represents  supply 
as  a  dynamic  interplay  rather  than  a  specific  model,  survey,  or  analysis. 


DEVELOPMENT  OF  COMPUTER-BASED  WRITING  AIDS1 

Frederick  R.  Chang 


|  Background 

Increasing  national  interest  in  the  quality  of  written  communication  is  evidenced  by 
the  number  of  state  legislatures  who  have  enacted  requirements  for  plain  English  in  public 
documents  and  contracts.  Government  writing  ("governmentese"),  in  particular,  has  been 
(  the  target  of  much  action.  Recently,  for  example,  a  federal  judge  in  Brooklyn,  New  York, 

I  ordered  the  government  to  remove  the  gobbledygook  from  Medicare  form  letters.  In  a 

verdict  on  a  class-action  suit,  the  judge  ruled  that  the  language  used  in  Medicare  letters 
could  not  be  understood  by  the  recipients  and,  in  fact,  did  not  qualify  as  English  (Simply 
Stated,  1984). 

There  is  no  shortage  of  guidelines  for  improving  either  government  or  Department  of 
j  Defense  writing,  yet  the  problems  persist.  Either  the  guidelines  are  not  being  read  and 

followed,  the  procedures  are  too  difficult  and  time-consuming  to  implement,  or  the 
suggestions  themselves  are  not  adequate. 

As  word  processors  and  computer  systems  are  gaining  widespread  acceptance,  it 
appears  the  potential  now  exists  to  develop  computerized  writing  assistants.  A  variety  of 
automated  writing  tools,  such  as  spell  checkers  and  text  formatters,  already  exist,  but  the 
potential  exists  for  more  advanced  tools.  The  few  automated  tools  that  are  available  tend 
to  be  lacking  in  two  important  respects:  First,  these  writing  tools  should  incorporate 
findings  from  the  psycholinguistic  literature.  Much  is  known  about  how  people 
comprehend  sentences,  paragraphs,  and  texts  in  general,  and  it  is  important  to  use  this 
knowledge  (Kieras,  1984).  Second,  it  is  important  to  provide  the  writer  with  an  efficient 
I  interface  to  the  computer  (cf.  Norman,  1981).  Current  programs  have  not  provided  clean, 

efficient  user  interfaces. 

Objective 

This  effort  concentrated  on  the  user  interface.  The  objective  was  to  develop  an 
efficient,  easy-to-use,  computer-based  writing  assistant  that  helps  a  writer  follow  the 
"plain  English"  guidelines  specified  in  the  recent  edition  of  the  Navy  Correspondence 
Manual  (Secretary  of  the  Navy,  1983).  The  first  chapter  of  this  Navy  instruction  details 
fairly  concrete  steps  to  improve  writing,  consistent  with  plain  English  initiative.  The 
ability  to  implement  these  guidelines  easily  is  potentially  of  great  importance  to  the 
effectivness  of  Navy  communication. 

Approach 

A  set  of  design  principles  was  determined  to  be  crucial  to  an  effective  computer- 
based  writing  assistant:  The  system  should  be  (1)  easy  to  use,  (2)  provide  specific 
information  about  what  is  wrong  with  the  document,  (3)  provide  specific  recommendations 
about  how  to  improve  the  document,  and  (4)  have  a  quick  and  inexpensive  user  interface. 
With  these  guidelines  in  mind,  the  REVISE  system  was  developed. 


The  Navy  instruction  includes  a  discussion  of  the  importance  of  using  simpler  words 
and  phrases  and,  in  general,  the  importance  of  economy  in  writing.  The  instruction  also 
includes  a  "hit  list"  of  excessively  wordy  or  commonly  misused  phrases  along  with 
suggestions  for  improving  them.  This  listing  was  the  foundation  of  the  data  base  in 
REVISE.  The  items  were  modified  and  enhanced  for  use  in  the  system. 


Operation 

After  the  writer  has  prepared  a  draft  document,  he  or  she  can  then  run  the  document 
through  REVISE.  The  program  runs  interactively  and  analyzes  each  sentence  separately. 
If  the  system  detects  a  word  or  phrase  contained  in  the  data  base  of  wordy  or  unclear 
expressions,  it  displays  the  entire  sentence  and  highlights  the  awkward  part.  It  then 
displays  alternative  words  or  phrases  that  could  replace  the  flagged  items.  The  system 
also  displays  a  menu  of  options  as  a  quick,  easy  way  to  make  changes  to  the  document. 
Once  the  revision  is  complete,  the  system  creates  a  file  (on  disk)  that  contains  all 
changes.  The  original  file  is  left  intact. 

Results 

A  collection  of  Navy  documents  was  run  through  the  system,  and  changes  suggested 
by  REVISE  were  made.  In  general,  over  90  percent  of  the  awkward  words  and  phrases 
flagged  by  the  system  were  appropriate  targets  for  revision  according  to  the  sentence 
context.  The  original  and  revised  documents  were  compared  on  a  variety  of  text 
characteristics.  The  revised  documents  showed  significant  improvements  on  a  variety  of 
measures  thought  to  influence  comprehensibility.  For  example,  the  revised  documents 
generally  had  more  common  (higher  frequency)  words,  fewer  passive  constructions,  fewer 
nominalizations,  improved  readability  scores,  and  the  like.  There  were  no  significant 
changes  in  sentence  construction  measurements. 

Conclusions 

Although  a  full-scale  comprehensibility  evaluation  was  beyond  the  scope  of  this 
research,  REVISE  did  make  measurable  changes  in  a  collection  of  Navy  documents  in  ways 
that  are  known  to  improve  comprehensibility.  The  system  described  here  represents  an 
example  of  a  computerized  writing  assistant  that  can  help  military  personnel  to  write 
plain  English.  Although  REVISE  improved  Navy  documents  in  some  important  ways,  the 
improvements  were  limited  in  other  ways.  A  variety  of  other  computer-based  writing 
tools  that  are  now  under  development  will  hopefully  provide  increased  capability. 
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1.  The  FY83  and  FY84  results  of  the  work  under  this  project  title  (ZROOO-0 1-042- 
04.014)  will  provide  the  basis  for  evaluating  the  effects  of  changing  the  delivery  of  the 
Navy  Basic  Electricity  and  Electronic  Course  from  self-paced  to  group-paced/lecture 
based  instruction.  The  CHANGE  diagnostic  test  developed  for  this  project  is  being  used 
for  an  ONR  supported  research  effort  to  test  the  usefulness  of  adaptive  testing  models  for 
instruction. 

2.  Work  performed  under  the  title  "Generalizable  Cognitive  Strategies”  (ZR000-01-042- 
06.017)  was  completed  in  FY84  and  transitioned  to  Exploratory  Development  task 
PE62763N,  RF63622801;  "Generic  Problem  Solving." 

3.  Some  of  the  work  undertaken  in  FY84  under  the  project  title  'Cognitive  Factors  in 
Learning  and  Retention"  (ZROOO-0 1-042-06.027)  will  be  used  for  Advanced  Development 
tasks  PE63720N;  Z 1772-002:  Functional  Context  Training,  and  PE62763N;  RF63522801: 
Classroom  Instructional  Process. 

4.  "Acquiring  Skill  from  Work  Experience"  (ZROOO-0 1-042-08.023)  have  led  to  the 
proposal  of  new  FY87  6.2  level  research  in  officer  career  development.  That  research 
would  look  at  both  the  optimum  length  of  operational  (sea),  quasi-  operational  (sea-related 
shore),  and  nonoperational  (unrelated  shore)  tours  and  the  sequences  of  tour  assignments 
required  to  produce  effective  operational  and  nonoperational  commanding  officers. 
Support  for  such  a  proposal  has  been  received  from  OP- 1 3. 

5.  Results  from  the  project  titled  "Models  of  Human  Performance  with  Application  to 
Decision  Aiding"  (ZROOO-0 1-042-09.022)  have  been  incorporated  into  Exploratory  Develop¬ 
ment  Task  PE62757N;  SF57525601:  "Decision  Aiding  in  Combat  Systems." 


INDEPENDENT  EXPLORATORY  DEVELPMENT  PROJECTS 

1.  The  results  of  work  under  the  project  title  "Multiple  Criterion  Optimization 
Techniques"  (ZF66-51 1.013)  have  influenced  work  in  three  FY85-funded  efforts:  (a) 
PE63707N,  Z1770:  Personnel  Assignment  System,  (b)  PE62763N,  RF63-52 1-004-031: 
Career  Management  Planning,  and  (c)  PE99000N,  C0073-002.02:  CLASP  System  Develop¬ 
ment. 

2.  Work  under  the  project  title  "Self-Defense  Tactical  Trainer"  (ZF66-512-001.008)  has 
been  transitioned  to  Advanced  Development  task  PE63720N,  Z1772:  Tactical  Decision 
T  raining. 

3.  ZF66-512-014:  Development  of  Computer-based  Writing  Aids.  This  work  has 
transitioned  to  task  PE63720N,  Z1772:  Authoring  Instructional  Materials.  Results  of  the 
FY84  work  will  be  applied  to  the  revision  of  the  NATO  Sea  Sparrow  missile  system 
technical  manual  in  a  project  sponsored  by  the  NAVSEA  NATO  Sea  Sparrow  Project 
Office  (C06P). 
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